[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:28:44 dist-test-slave-1xrh krb5kdc[6761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:28:44 dist-test-slave-1xrh krb5kdc[6761](info): set up 2 sockets
Aug 06 12:28:44 dist-test-slave-1xrh krb5kdc[6761](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 12:28:45.478572 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:28:45 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:28:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:28:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:28:45.600816 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:41249
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:41249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:45.714160 6774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:45.714416 6774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:45.714454 6774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:45.718531 6774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:28:45.718590 6774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:45.718611 6774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:45.718627 6774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:28:45.718641 6774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:28:45.724237 6774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:41249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:41249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:45.725507 6774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:45.726728 6774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:45.739868 6781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:45.740806 6780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:45.741180 6783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:45.741420 6774 server_base.cc:1034] running on GCE node
I20260806 12:28:45.742138 6774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:45.743997 6774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:45.745180 6774 hybrid_clock.cc:648] HybridClock initialized: now 1786019325745141 us; error 76 us; skew 500 ppm
Aug 06 12:28:45 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:45.748665 6774 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:28:45.749730 6774 webserver.cc:460] Webserver started at http://127.6.152.190:43141/ using document root <none> and password file <none>
I20260806 12:28:45.750599 6774 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:45.750700 6774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:45.751106 6774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:45.753893 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "c47913030b864cf689b08befe74c1ed4"
format_stamp: "Formatted at 2026-08-06 12:28:45 on dist-test-slave-1xrh"
server_key: "0b8aef865ca2caad4276091f2b1f3980"
server_key_iv: "16711b4800ad1e1e82216baa8c89f135"
server_key_version: "encryptionkey@0"
I20260806 12:28:45.754498 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "c47913030b864cf689b08befe74c1ed4"
format_stamp: "Formatted at 2026-08-06 12:28:45 on dist-test-slave-1xrh"
server_key: "0b8aef865ca2caad4276091f2b1f3980"
server_key_iv: "16711b4800ad1e1e82216baa8c89f135"
server_key_version: "encryptionkey@0"
I20260806 12:28:45.759438 6774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 12:28:45.762894 6790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:45.764518 6774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:28:45.764707 6774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "c47913030b864cf689b08befe74c1ed4"
format_stamp: "Formatted at 2026-08-06 12:28:45 on dist-test-slave-1xrh"
server_key: "0b8aef865ca2caad4276091f2b1f3980"
server_key_iv: "16711b4800ad1e1e82216baa8c89f135"
server_key_version: "encryptionkey@0"
I20260806 12:28:45.764827 6774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:45.789454 6774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:45.793448 6774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:45.793756 6774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:45.810356 6774 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:41249
I20260806 12:28:45.810539 6848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:41249 every 8 connection(s)
I20260806 12:28:45.811846 6774 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:28:45.816691 6849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:45.817490 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 6774
I20260806 12:28:45.817631 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:28:45.818988 6754 external_mini_cluster.cc:1428] Setting key 21a0c5ac7688e087685c2335013513aa
I20260806 12:28:45.827268 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4: Bootstrap starting.
I20260806 12:28:45.831954 6849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:45.833385 6849 log.cc:824] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:45.836171 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4: No bootstrap required, opened a new log
Aug 06 12:28:45 dist-test-slave-1xrh krb5kdc[6761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:45.841375 6849 raft_consensus.cc:348] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47913030b864cf689b08befe74c1ed4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41249 } }
I20260806 12:28:45.841771 6849 raft_consensus.cc:374] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:45.841910 6849 raft_consensus.cc:729] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c47913030b864cf689b08befe74c1ed4, State: Initialized, Role: FOLLOWER
I20260806 12:28:45.842739 6849 consensus_queue.cc:260] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47913030b864cf689b08befe74c1ed4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41249 } }
I20260806 12:28:45.842960 6849 raft_consensus.cc:388] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:28:45.843114 6849 raft_consensus.cc:482] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:28:45.843321 6849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:45.844300 6849 raft_consensus.cc:504] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47913030b864cf689b08befe74c1ed4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41249 } }
I20260806 12:28:45.844902 6849 leader_election.cc:302] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c47913030b864cf689b08befe74c1ed4; no voters:
I20260806 12:28:45.845327 6849 leader_election.cc:288] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:28:45.845543 6854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:45.845903 6854 raft_consensus.cc:686] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [term 1 LEADER]: Becoming Leader. State: Replica: c47913030b864cf689b08befe74c1ed4, State: Running, Role: LEADER
I20260806 12:28:45.846415 6854 consensus_queue.cc:237] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47913030b864cf689b08befe74c1ed4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41249 } }
I20260806 12:28:45.847281 6849 sys_catalog.cc:564] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:28:45.847412 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:45.823559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.6.152.190:41249)
0806 12:28:45.824081 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:45.824121 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:45.825462 (+ 1341us) server_negotiation.cc:407] Connection header received
0806 12:28:45.830062 (+ 4600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:45.830077 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:45.830457 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:45.830770 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:45.833445 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:45.834703 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:45.836048 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:45.836531 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:45.836697 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:45.841025 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:45.841059 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:45.841070 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:45.841097 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:45.843531 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:45.844060 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:45.844066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:45.844072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:45.844190 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:45.844599 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:45.844602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:45.844604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:45.845129 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:45.845367 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:45.846095 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 12:28:45.846391 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":141,"threads_started":1}
I20260806 12:28:45.851694 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c47913030b864cf689b08befe74c1ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47913030b864cf689b08befe74c1ed4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41249 } } }
I20260806 12:28:45.851848 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:45.852083 6855 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c47913030b864cf689b08befe74c1ed4. Latest consensus state: current_term: 1 leader_uuid: "c47913030b864cf689b08befe74c1ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47913030b864cf689b08befe74c1ed4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41249 } } }
I20260806 12:28:45.852522 6855 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:45.852298 6869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:28:45.857060 6869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:28:45.864523 6869 catalog_manager.cc:1347] Generated new cluster ID: bbf9110f58964768a26e5619a85f499e
I20260806 12:28:45.864589 6869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:28:45.890014 6869 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:28:45.891340 6869 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:28:45.900101 6869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c47913030b864cf689b08befe74c1ed4: Generated new TSK 0
I20260806 12:28:45.901129 6869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:28:45.936414 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:41249
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:46.080185 6875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:46.080447 6875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:46.080492 6875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:46.084659 6875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:46.084718 6875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:46.084815 6875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:28:46.090173 6875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:41249
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:46.091287 6875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:46.092293 6875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:46.102743 6880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:46.102824 6881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:46.103004 6875 server_base.cc:1034] running on GCE node
W20260806 12:28:46.111459 6884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:46.111979 6875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:46.112720 6875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:46.113916 6875 hybrid_clock.cc:648] HybridClock initialized: now 1786019326113885 us; error 40 us; skew 500 ppm
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:46.117309 6875 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:28:46.118173 6875 webserver.cc:460] Webserver started at http://127.6.152.129:46679/ using document root <none> and password file <none>
I20260806 12:28:46.118747 6875 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:46.118844 6875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:46.119222 6875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:46.121506 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "67e16c5f0ab94bcfa7b72a4d059e3cdc"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "fc94ca9755a1d8917dbf950297fafd81"
server_key_iv: "e99803834a3da4ebffe7b71080b34158"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.122138 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "67e16c5f0ab94bcfa7b72a4d059e3cdc"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "fc94ca9755a1d8917dbf950297fafd81"
server_key_iv: "e99803834a3da4ebffe7b71080b34158"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.126237 6875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 12:28:46.128932 6892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.130163 6875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:46.130311 6875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "67e16c5f0ab94bcfa7b72a4d059e3cdc"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "fc94ca9755a1d8917dbf950297fafd81"
server_key_iv: "e99803834a3da4ebffe7b71080b34158"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.130441 6875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:46.148226 6875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:46.151862 6875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:46.152091 6875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:46.152923 6875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:28:46.154001 6875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:46.154075 6875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.154150 6875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:46.154201 6875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.171283 6875 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:38459
I20260806 12:28:46.171439 7013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:38459 every 8 connection(s)
I20260806 12:28:46.172254 6875 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:28:46.173319 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 6875
I20260806 12:28:46.173439 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:28:46.173728 6754 external_mini_cluster.cc:1428] Setting key d6bee0bd7f8bf2bb5795bf28bdd0d7ab
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:28:46.189942 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.174598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:37451 (local address 127.6.152.190:41249)
0806 12:28:46.174746 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:46.174749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:46.175837 (+ 1088us) server_negotiation.cc:407] Connection header received
0806 12:28:46.177839 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:46.177842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:46.177888 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:46.177962 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:46.179872 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.180382 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.182444 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.182664 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.182741 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.185965 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:46.185968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:46.185970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:46.185994 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:46.187686 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.188295 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.188299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.188300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.188436 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.188843 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.188846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.188848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.189032 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:46.189145 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:46.189637 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 12:28:46.189762 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:28:46.190734 7016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.175088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:41249 (local address 127.6.152.129:37451)
0806 12:28:46.175693 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:46.175734 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:46.177599 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:46.178167 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:46.178178 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:46.178837 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:46.179715 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:46.179738 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.180570 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.180577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:46.182306 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:46.182333 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.182849 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.182856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:46.183208 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.184006 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:46.184030 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:46.185790 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:46.187813 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:46.187819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:46.187833 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:46.188179 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:46.188537 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:46.188542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:46.188544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:46.188737 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:46.189144 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:46.189151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:46.189392 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 12:28:46.189579 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 12:28:46.189891 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":190,"threads_started":1}
I20260806 12:28:46.192152 7014 heartbeater.cc:347] Connected to a master server at 127.6.152.190:41249
I20260806 12:28:46.192440 7014 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:46.192921 7014 heartbeater.cc:511] Master 127.6.152.190:41249 requested a full tablet report, sending...
I20260806 12:28:46.194814 6810 ts_manager.cc:194] Registered new tserver with Master: 67e16c5f0ab94bcfa7b72a4d059e3cdc (127.6.152.129:38459)
I20260806 12:28:46.196271 6810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:37451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:28:46.204092 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:41249
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:46.338253 7019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:46.338546 7019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:46.338637 7019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:46.343389 7019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:46.343483 7019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:46.343577 7019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:28:46.349372 7019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:41249
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:46.350447 7019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:46.351595 7019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:46.362689 7025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:46.362722 7026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:46.362722 7028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:46.363070 7019 server_base.cc:1034] running on GCE node
I20260806 12:28:46.363536 7019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:46.364245 7019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:46.365430 7019 hybrid_clock.cc:648] HybridClock initialized: now 1786019326365413 us; error 29 us; skew 500 ppm
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:46.373760 7019 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:28:46.374603 7019 webserver.cc:460] Webserver started at http://127.6.152.130:34951/ using document root <none> and password file <none>
I20260806 12:28:46.375244 7019 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:46.375314 7019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:46.375487 7019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:46.377578 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "6a723022e047452a8fbbceb1103dba59"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "8964e2baf2b26ad6cba1f1a5fc9e05e3"
server_key_iv: "7d5d5181d44bc47a731fd6f03742c547"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.378329 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "6a723022e047452a8fbbceb1103dba59"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "8964e2baf2b26ad6cba1f1a5fc9e05e3"
server_key_iv: "7d5d5181d44bc47a731fd6f03742c547"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.382450 7019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 12:28:46.385056 7035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.386263 7019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:46.386385 7019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "6a723022e047452a8fbbceb1103dba59"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "8964e2baf2b26ad6cba1f1a5fc9e05e3"
server_key_iv: "7d5d5181d44bc47a731fd6f03742c547"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.386503 7019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:46.406793 7019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:46.410360 7019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:46.410584 7019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:46.411257 7019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:28:46.412120 7019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:46.412169 7019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.412247 7019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:46.412281 7019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.428896 7019 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:46855
I20260806 12:28:46.429018 7155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:46855 every 8 connection(s)
I20260806 12:28:46.429890 7019 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:28:46.430112 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7019
I20260806 12:28:46.430214 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:28:46.430549 6754 external_mini_cluster.cc:1428] Setting key a34ec890d89840fce18bdb8fd6b42fc9
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:28:46.448081 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.432137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:41569 (local address 127.6.152.190:41249)
0806 12:28:46.432307 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:46.432312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:46.433556 (+ 1244us) server_negotiation.cc:407] Connection header received
0806 12:28:46.435443 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:46.435447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:46.435531 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:46.435635 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:46.436945 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.437492 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.439804 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.440037 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.440121 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.444053 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:46.444056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:46.444058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:46.444082 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:46.445654 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.446178 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.446181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.446183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.446267 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.446665 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.446667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.446668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.446879 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:46.446992 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:46.447765 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 12:28:46.447938 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:28:46.449218 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.432664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:41249 (local address 127.6.152.130:41569)
0806 12:28:46.433377 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:46.433432 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:46.435197 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:46.435832 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:46.435840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:46.436260 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:46.436804 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:46.436819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.437651 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.437658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:46.439667 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:46.439683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.440221 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.440228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:46.440577 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.441367 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:46.441397 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:46.443822 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:46.445782 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:46.445788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:46.445802 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:46.446065 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:46.446363 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:46.446367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:46.446369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:46.446507 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:46.446991 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:46.446998 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:46.447364 (+ 366us) client_negotiation.cc:769] Sending connection context
0806 12:28:46.447656 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 12:28:46.448011 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":224,"threads_started":1}
I20260806 12:28:46.450546 7156 heartbeater.cc:347] Connected to a master server at 127.6.152.190:41249
I20260806 12:28:46.450809 7156 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:46.451311 7156 heartbeater.cc:511] Master 127.6.152.190:41249 requested a full tablet report, sending...
I20260806 12:28:46.452303 6810 ts_manager.cc:194] Registered new tserver with Master: 6a723022e047452a8fbbceb1103dba59 (127.6.152.130:46855)
I20260806 12:28:46.452911 6810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:41569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:28:46.465576 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:41249
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:46.609849 7161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:46.610116 7161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:46.610193 7161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:46.614080 7161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:46.614161 7161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:46.614248 7161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:28:46.619663 7161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:41249
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:46.620760 7161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:46.621829 7161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:46.632507 7169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:46.632470 7166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:46.632529 7167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:46.632699 7161 server_base.cc:1034] running on GCE node
I20260806 12:28:46.633054 7161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:46.633693 7161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:46.634865 7161 hybrid_clock.cc:648] HybridClock initialized: now 1786019326634831 us; error 46 us; skew 500 ppm
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:46.637955 7161 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:28:46.638726 7161 webserver.cc:460] Webserver started at http://127.6.152.131:41443/ using document root <none> and password file <none>
I20260806 12:28:46.639348 7161 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:46.639433 7161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:46.639658 7161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:46.641876 7161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "3672f83c470d490e841f1ac5e5c04c5d"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "6cae9a2371626fd909d5f142862eb89c"
server_key_iv: "fe233ccc1fa35805c8d8fe6b61bbe022"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.642445 7161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "3672f83c470d490e841f1ac5e5c04c5d"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "6cae9a2371626fd909d5f142862eb89c"
server_key_iv: "fe233ccc1fa35805c8d8fe6b61bbe022"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.646136 7161 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:28:46.648465 7176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.649838 7161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:46.649977 7161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "3672f83c470d490e841f1ac5e5c04c5d"
format_stamp: "Formatted at 2026-08-06 12:28:46 on dist-test-slave-1xrh"
server_key: "6cae9a2371626fd909d5f142862eb89c"
server_key_iv: "fe233ccc1fa35805c8d8fe6b61bbe022"
server_key_version: "encryptionkey@0"
I20260806 12:28:46.650090 7161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:46.670077 7161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:46.673606 7161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:46.673868 7161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:46.674688 7161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:28:46.675786 7161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:46.675866 7161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.675937 7161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:46.675985 7161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:46.692516 7161 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35585
I20260806 12:28:46.692539 7292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35585 every 8 connection(s)
I20260806 12:28:46.693531 7161 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:28:46.702430 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7161
I20260806 12:28:46.702536 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:28:46.702821 6754 external_mini_cluster.cc:1428] Setting key 4684b0095b4845f323ffdb68ac0492b6
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:46.707573 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.695489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:43249 (local address 127.6.152.190:41249)
0806 12:28:46.695652 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:46.695658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:46.696558 (+ 900us) server_negotiation.cc:407] Connection header received
0806 12:28:46.697857 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:46.697859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:46.697913 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:46.697988 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:46.699211 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.699740 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.700879 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.701106 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.701184 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.704025 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:46.704029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:46.704032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:46.704065 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:46.705504 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.706027 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.706031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.706033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.706116 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.706460 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.706464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.706466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.706646 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:46.706755 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:46.707275 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 12:28:46.707391 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 12:28:46.708341 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.695834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:41249 (local address 127.6.152.131:43249)
0806 12:28:46.696400 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:46.696442 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:46.697663 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:46.698155 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:46.698161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:46.698522 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:46.699023 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:46.699044 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.699850 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.699853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:46.700771 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:46.700781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.701251 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.701254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:46.701407 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.701957 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:46.701973 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:46.703824 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:46.705635 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:46.705641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:46.705654 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:46.705920 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:46.706213 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:46.706217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:46.706218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:46.706356 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:46.706755 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:46.706762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:46.707001 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 12:28:46.707228 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 12:28:46.707538 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":213,"threads_started":1}
I20260806 12:28:46.709506 7293 heartbeater.cc:347] Connected to a master server at 127.6.152.190:41249
I20260806 12:28:46.709769 7293 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:46.710234 7293 heartbeater.cc:511] Master 127.6.152.190:41249 requested a full tablet report, sending...
I20260806 12:28:46.711309 6810 ts_manager.cc:194] Registered new tserver with Master: 3672f83c470d490e841f1ac5e5c04c5d (127.6.152.131:35585)
I20260806 12:28:46.712234 6810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:43249
I20260806 12:28:46.717003 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:28:46.747623 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.737391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.6.152.129:38459)
0806 12:28:46.737832 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:46.737839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:46.737909 (+ 70us) server_negotiation.cc:407] Connection header received
0806 12:28:46.738044 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:46.738049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:46.738238 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:46.738402 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:46.739164 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.739917 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.740754 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.741180 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.741268 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.743900 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:46.743910 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:46.743923 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:46.743946 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:46.745675 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.746141 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.746146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.746147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.746227 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.746524 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.746527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.746529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.746779 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:46.747013 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:46.747257 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 12:28:46.747437 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":155,"threads_started":1}
W20260806 12:28:46.748643 6921 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50540
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[6761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:28:46.779649 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:46.770441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.6.152.129:38459)
0806 12:28:46.770586 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:46.770591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:46.770861 (+ 270us) server_negotiation.cc:407] Connection header received
0806 12:28:46.770972 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:46.770975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:46.771015 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:46.771084 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:46.771785 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.772318 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.773071 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:46.773300 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:46.773372 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:46.776517 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:46.776520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:46.776522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:46.776543 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:46.778017 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.778418 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.778421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.778422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.778492 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:46.778727 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:46.778729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:46.778730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:46.778874 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:46.778979 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:46.779291 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 12:28:46.779419 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:28:46.780907 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 6875
I20260806 12:28:46.788476 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7019
I20260806 12:28:46.794361 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7161
I20260806 12:28:46.800019 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 6774
2026-08-06T12:28:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2421 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[7308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[7308](info): set up 2 sockets
Aug 06 12:28:46 dist-test-slave-1xrh krb5kdc[7308](info): commencing operation
krb5kdc: starting...
W20260806 12:28:47.865729 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:28:47 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:28:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:28:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:28:47.982365 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:42789
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:42789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:48.092919 7321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:48.093205 7321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:48.093291 7321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:48.097227 7321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:28:48.097308 7321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:48.097347 7321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:48.097406 7321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:28:48.097445 7321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:28:48.102461 7321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:42789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:42789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:48.103497 7321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:48.104512 7321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:48.112705 7327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.112715 7328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.112739 7330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:48.112952 7321 server_base.cc:1034] running on GCE node
I20260806 12:28:48.113492 7321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:48.114503 7321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:48.115689 7321 hybrid_clock.cc:648] HybridClock initialized: now 1786019328115671 us; error 29 us; skew 500 ppm
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:48.118394 7321 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:28:48.119284 7321 webserver.cc:460] Webserver started at http://127.6.152.190:36385/ using document root <none> and password file <none>
I20260806 12:28:48.119835 7321 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:48.119928 7321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:48.120172 7321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:48.122134 7321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "ea25a3a3887342b69fcb3afab695c7d7"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "9c4832d40dc5cebcb5b22c4f02a1420e"
server_key_iv: "d487fa73a82a8ef6430640fa4f71a76f"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.122697 7321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "ea25a3a3887342b69fcb3afab695c7d7"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "9c4832d40dc5cebcb5b22c4f02a1420e"
server_key_iv: "d487fa73a82a8ef6430640fa4f71a76f"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.126300 7321 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 12:28:48.128676 7338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.129949 7321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:48.130098 7321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "ea25a3a3887342b69fcb3afab695c7d7"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "9c4832d40dc5cebcb5b22c4f02a1420e"
server_key_iv: "d487fa73a82a8ef6430640fa4f71a76f"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.130213 7321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:48.156184 7321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:48.158959 7321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:48.159237 7321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:48.166421 7321 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:42789
I20260806 12:28:48.166450 7393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:42789 every 8 connection(s)
I20260806 12:28:48.167444 7321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:28:48.168650 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7321
I20260806 12:28:48.168764 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:28:48.169059 6754 external_mini_cluster.cc:1428] Setting key b66218fe27efe4969f980665288b6824
I20260806 12:28:48.170903 7394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:48.177508 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7: Bootstrap starting.
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:48.180239 7394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:48.181145 7394 log.cc:824] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:48.183468 7394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7: No bootstrap required, opened a new log
I20260806 12:28:48.184158 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.170614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.6.152.190:42789)
0806 12:28:48.171100 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.171116 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.171207 (+ 91us) server_negotiation.cc:407] Connection header received
0806 12:28:48.171867 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.171883 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.172185 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.172496 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:48.173395 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.174387 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.175634 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.176075 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.176198 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.178987 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:48.178994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:48.179010 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:48.179041 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:48.181051 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.181531 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.181540 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.181549 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.181683 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.182019 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.182022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.182023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.182502 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:48.182697 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.182903 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.183232 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":152,"threads_started":1}
I20260806 12:28:48.187693 7394 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea25a3a3887342b69fcb3afab695c7d7" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42789 } }
I20260806 12:28:48.187908 7394 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:48.187944 7394 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea25a3a3887342b69fcb3afab695c7d7, State: Initialized, Role: FOLLOWER
I20260806 12:28:48.188287 7394 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea25a3a3887342b69fcb3afab695c7d7" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42789 } }
I20260806 12:28:48.188396 7394 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:28:48.188441 7394 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:28:48.188508 7394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:48.189325 7394 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea25a3a3887342b69fcb3afab695c7d7" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42789 } }
I20260806 12:28:48.189616 7394 leader_election.cc:302] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea25a3a3887342b69fcb3afab695c7d7; no voters:
I20260806 12:28:48.189864 7394 leader_election.cc:288] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:28:48.190049 7400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:48.190368 7400 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [term 1 LEADER]: Becoming Leader. State: Replica: ea25a3a3887342b69fcb3afab695c7d7, State: Running, Role: LEADER
I20260806 12:28:48.190704 7400 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea25a3a3887342b69fcb3afab695c7d7" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42789 } }
I20260806 12:28:48.191051 7394 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:28:48.192312 7402 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea25a3a3887342b69fcb3afab695c7d7. Latest consensus state: current_term: 1 leader_uuid: "ea25a3a3887342b69fcb3afab695c7d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea25a3a3887342b69fcb3afab695c7d7" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42789 } } }
I20260806 12:28:48.192444 7402 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:48.192790 7401 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea25a3a3887342b69fcb3afab695c7d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea25a3a3887342b69fcb3afab695c7d7" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42789 } } }
I20260806 12:28:48.192888 7401 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:48.192937 7410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:28:48.195791 7410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:28:48.200500 7410 catalog_manager.cc:1347] Generated new cluster ID: a7eabcb08f5a430baf54a5ef08417661
I20260806 12:28:48.200587 7410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:28:48.212268 7410 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:28:48.213128 7410 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:28:48.217335 7410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea25a3a3887342b69fcb3afab695c7d7: Generated new TSK 0
I20260806 12:28:48.217935 7410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:28:48.252324 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42789
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:48.363356 7423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:48.363617 7423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:48.363692 7423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:48.367514 7423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:48.367594 7423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:48.367698 7423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:28:48.372849 7423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:48.373857 7423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:48.374848 7423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:48.384197 7428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.384187 7429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.384167 7431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:48.384585 7423 server_base.cc:1034] running on GCE node
I20260806 12:28:48.384969 7423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:48.385597 7423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:48.386762 7423 hybrid_clock.cc:648] HybridClock initialized: now 1786019328386744 us; error 30 us; skew 500 ppm
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:48.389475 7423 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:28:48.390348 7423 webserver.cc:460] Webserver started at http://127.6.152.129:39651/ using document root <none> and password file <none>
I20260806 12:28:48.390904 7423 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:48.391000 7423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:48.391275 7423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:48.393422 7423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "21d878161b7d4233aef424f40537e380"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "0a5773958d0e22508b32aef8ffd09c58"
server_key_iv: "513ca4790888dd5ff9c1b7ab87db5b29"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.393985 7423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "21d878161b7d4233aef424f40537e380"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "0a5773958d0e22508b32aef8ffd09c58"
server_key_iv: "513ca4790888dd5ff9c1b7ab87db5b29"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.397552 7423 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:28:48.399891 7438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.401016 7423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:48.401171 7423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "21d878161b7d4233aef424f40537e380"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "0a5773958d0e22508b32aef8ffd09c58"
server_key_iv: "513ca4790888dd5ff9c1b7ab87db5b29"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.401281 7423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:48.418829 7423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:48.421679 7423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:48.421893 7423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:48.422505 7423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:28:48.423394 7423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:48.423467 7423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.423537 7423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:48.423589 7423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.432956 7423 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:46823
I20260806 12:28:48.432984 7555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:46823 every 8 connection(s)
I20260806 12:28:48.433911 7423 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:28:48.438776 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7423
I20260806 12:28:48.438899 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:28:48.439208 6754 external_mini_cluster.cc:1428] Setting key 207d59bfa724087aa11884d2d5fab672
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:48.449565 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.435725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:48609 (local address 127.6.152.190:42789)
0806 12:28:48.435936 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.435940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.436765 (+ 825us) server_negotiation.cc:407] Connection header received
0806 12:28:48.438160 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.438162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.438210 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.438284 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:48.439859 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.440462 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.442258 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.442494 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.442639 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.445485 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:48.445491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:48.445497 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:48.445533 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:48.447328 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.447848 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.447853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.447856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.447950 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.448333 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.448337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.448338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.448619 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:48.448730 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.449215 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.449354 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 12:28:48.450253 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.436161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:42789 (local address 127.6.152.129:48609)
0806 12:28:48.436605 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:48.436648 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:48.437960 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:48.438441 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:48.438448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:48.438966 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:48.439677 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.439698 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.440660 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.440666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.442120 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.442135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.442694 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.442700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.442904 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.443570 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:48.443586 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:48.445276 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:48.447461 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:48.447466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:48.447477 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:48.447739 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:48.448054 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:48.448056 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:48.448057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:48.448196 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:48.448726 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:48.448731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:48.448941 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 12:28:48.449090 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 12:28:48.449362 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:28:48.451511 7556 heartbeater.cc:347] Connected to a master server at 127.6.152.190:42789
I20260806 12:28:48.451717 7556 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:48.452314 7556 heartbeater.cc:511] Master 127.6.152.190:42789 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:28:48.453851 7356 ts_manager.cc:194] Registered new tserver with Master: 21d878161b7d4233aef424f40537e380 (127.6.152.129:46823)
I20260806 12:28:48.455428 7356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:48609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:28:48.469810 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42789
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:48.579838 7561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:48.580080 7561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:48.580121 7561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:48.584097 7561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:48.584161 7561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:48.584255 7561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:28:48.589440 7561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:48.590488 7561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:48.591629 7561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:48.601016 7566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.601061 7567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.601019 7569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:48.601261 7561 server_base.cc:1034] running on GCE node
I20260806 12:28:48.601728 7561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:48.602396 7561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:48.603580 7561 hybrid_clock.cc:648] HybridClock initialized: now 1786019328603567 us; error 25 us; skew 500 ppm
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:48.606475 7561 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:28:48.607331 7561 webserver.cc:460] Webserver started at http://127.6.152.130:43805/ using document root <none> and password file <none>
I20260806 12:28:48.607868 7561 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:48.607960 7561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:48.608198 7561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:48.610152 7561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "f0e24b050cb14dc4afc1153ec9a78f76"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "cc04ebd3e9e657cf08dfe1125529fd58"
server_key_iv: "eb9d264ea40ee0397f1c3e38f3bd2fc3"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.610713 7561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "f0e24b050cb14dc4afc1153ec9a78f76"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "cc04ebd3e9e657cf08dfe1125529fd58"
server_key_iv: "eb9d264ea40ee0397f1c3e38f3bd2fc3"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.614179 7561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:28:48.616490 7576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.617571 7561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:48.617712 7561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "f0e24b050cb14dc4afc1153ec9a78f76"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "cc04ebd3e9e657cf08dfe1125529fd58"
server_key_iv: "eb9d264ea40ee0397f1c3e38f3bd2fc3"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.617823 7561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:48.629825 7561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:48.632961 7561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:48.633155 7561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:48.633816 7561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:28:48.634701 7561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:48.634778 7561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.634831 7561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:48.634863 7561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.644032 7561 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:45063
I20260806 12:28:48.644060 7695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:45063 every 8 connection(s)
I20260806 12:28:48.644870 7561 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:28:48.645823 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7561
I20260806 12:28:48.645926 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:28:48.646235 6754 external_mini_cluster.cc:1428] Setting key e62ec1f9c3cc7de522f5cb387f03d772
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
I20260806 12:28:48.659551 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.646669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:41047 (local address 127.6.152.190:42789)
0806 12:28:48.646801 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.646806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.647746 (+ 940us) server_negotiation.cc:407] Connection header received
0806 12:28:48.649020 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.649023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.649081 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.649188 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:48.650251 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.650764 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.652561 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.652846 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.652934 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.655794 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:48.655797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:48.655799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:48.655821 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:48.657280 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.657834 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.657837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.657839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.657913 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.658264 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.658267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.658269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.658467 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:48.658593 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.659192 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.659317 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:28:48.660303 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.647060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:42789 (local address 127.6.152.130:41047)
0806 12:28:48.647601 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:48.647656 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:48.648820 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:48.649370 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:48.649376 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:48.649699 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:48.650104 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.650116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.650894 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.650901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.652398 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.652428 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.653005 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.653012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.653233 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.653840 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:48.653865 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:48.655610 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:48.657397 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:48.657403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:48.657416 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:48.657698 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:48.658011 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:48.658015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:48.658017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:48.658159 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:48.658594 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:48.658602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:48.658842 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 12:28:48.659083 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 12:28:48.659425 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":114,"threads_started":1}
I20260806 12:28:48.661607 7696 heartbeater.cc:347] Connected to a master server at 127.6.152.190:42789
I20260806 12:28:48.661857 7696 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:48.662367 7696 heartbeater.cc:511] Master 127.6.152.190:42789 requested a full tablet report, sending...
I20260806 12:28:48.663511 7356 ts_manager.cc:194] Registered new tserver with Master: f0e24b050cb14dc4afc1153ec9a78f76 (127.6.152.130:45063)
I20260806 12:28:48.664381 7356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:41047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:28:48.675540 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42789
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:48.784739 7701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:48.784978 7701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:48.785077 7701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:48.788900 7701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:48.788951 7701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:48.789065 7701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:28:48.794570 7701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:48.795573 7701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:48.796631 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:48.806305 7706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.806314 7709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:48.806362 7707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:48.806506 7701 server_base.cc:1034] running on GCE node
I20260806 12:28:48.807013 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:48.807736 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:48.808918 7701 hybrid_clock.cc:648] HybridClock initialized: now 1786019328808886 us; error 46 us; skew 500 ppm
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:48.811784 7701 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:28:48.812726 7701 webserver.cc:460] Webserver started at http://127.6.152.131:41735/ using document root <none> and password file <none>
I20260806 12:28:48.813282 7701 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:48.813373 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:48.813604 7701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:48.815649 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "80a414a54fae419baed3283f35bf9fcb"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "160a1617c1e8afa790fbad57ccb5fcde"
server_key_iv: "827d4048f6758d5abebf72c3469bcdc6"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.816228 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "80a414a54fae419baed3283f35bf9fcb"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "160a1617c1e8afa790fbad57ccb5fcde"
server_key_iv: "827d4048f6758d5abebf72c3469bcdc6"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.819729 7701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:28:48.822134 7717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.823400 7701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:48.823523 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "80a414a54fae419baed3283f35bf9fcb"
format_stamp: "Formatted at 2026-08-06 12:28:48 on dist-test-slave-1xrh"
server_key: "160a1617c1e8afa790fbad57ccb5fcde"
server_key_iv: "827d4048f6758d5abebf72c3469bcdc6"
server_key_version: "encryptionkey@0"
I20260806 12:28:48.823644 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:48.849386 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:48.852298 7701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:48.852450 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:48.852973 7701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:28:48.853710 7701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:48.853755 7701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.853792 7701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:48.853807 7701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:48.863000 7701 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:43655
I20260806 12:28:48.863040 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:43655 every 8 connection(s)
I20260806 12:28:48.863862 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:28:48.871619 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7701
I20260806 12:28:48.871709 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:28:48.872001 6754 external_mini_cluster.cc:1428] Setting key 3c203c3debc2858dbad1877de69fd6f4
Aug 06 12:28:48 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:48.877456 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.865540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58125 (local address 127.6.152.190:42789)
0806 12:28:48.865672 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.865676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.866447 (+ 771us) server_negotiation.cc:407] Connection header received
0806 12:28:48.867825 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.867829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.867941 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.868037 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:48.869220 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.869900 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.871023 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.871312 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.871426 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.874200 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:48.874204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:48.874208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:48.874239 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:48.875522 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.876020 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.876023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.876025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.876109 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.876443 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.876448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.876449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.876623 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:48.876728 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.877175 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.877294 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:28:48.878166 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.865864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:42789 (local address 127.6.152.131:58125)
0806 12:28:48.866284 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:48.866323 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:48.867591 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:48.868203 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:48.868209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:48.868548 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:48.869074 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.869087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.870028 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.870050 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.870889 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.870898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.871464 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.871468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.871631 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.872316 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:48.872346 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:48.874021 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:48.875639 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:48.875645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:48.875658 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:48.875904 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:48.876199 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:48.876202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:48.876204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:48.876342 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:48.876728 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:48.876734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:48.876953 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 12:28:48.877128 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 12:28:48.877421 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260806 12:28:48.879242 7839 heartbeater.cc:347] Connected to a master server at 127.6.152.190:42789
I20260806 12:28:48.879487 7839 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:48.879936 7839 heartbeater.cc:511] Master 127.6.152.190:42789 requested a full tablet report, sending...
I20260806 12:28:48.880870 7356 ts_manager.cc:194] Registered new tserver with Master: 80a414a54fae419baed3283f35bf9fcb (127.6.152.131:43655)
I20260806 12:28:48.881426 7356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:58125
I20260806 12:28:48.885807 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:28:48.896607 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.889673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.6.152.190:42789)
0806 12:28:48.889804 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.889808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.889994 (+ 186us) server_negotiation.cc:407] Connection header received
0806 12:28:48.890078 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.890081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.890118 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.890181 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:48.890842 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.891433 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.892302 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.892504 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.892586 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.893810 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:48.893813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:48.893814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:48.893835 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:48.895054 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.895469 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.895471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.895485 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.895554 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:48.895800 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:48.895803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:48.895804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:48.896021 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:48.896150 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.896339 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.896426 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:28:48.901825 7356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:28:48.905330 7356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:48.920877 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.914660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:45063 (local address 127.0.0.1:46652)
0806 12:28:48.914995 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:48.915045 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:48.915929 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:48.916707 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:48.916711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:48.916731 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:48.916889 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.916894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.917894 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.917899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.919089 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.919099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.920260 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.920264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.920384 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.920422 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 12:28:48.920544 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 12:28:48.920722 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":33,"thread_start_us":78,"threads_started":1}
I20260806 12:28:48.920871 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.914777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.6.152.130:45063)
0806 12:28:48.915454 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.915469 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.915497 (+ 28us) server_negotiation.cc:407] Connection header received
0806 12:28:48.916121 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.916130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.916347 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.916509 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:48.917014 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.917753 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.919288 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.919850 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.919980 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.920187 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.920590 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.920710 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":140,"threads_started":1}
I20260806 12:28:48.921365 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.915056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:43655 (local address 127.0.0.1:40450)
0806 12:28:48.915659 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:48.915674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:48.915807 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:48.916498 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:48.916504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:48.916526 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:48.916679 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.916689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.918288 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.918291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.919823 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.919835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.920948 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.920952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.921071 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.921093 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 12:28:48.921163 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 12:28:48.921228 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":128,"thread_start_us":118,"threads_started":1}
I20260806 12:28:48.921828 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.915585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.6.152.131:43655)
0806 12:28:48.916057 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.916073 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.916096 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:28:48.916168 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.916175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.916346 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.916506 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:48.916807 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.918127 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.919973 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.920589 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.920740 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.920877 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.921539 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.921633 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":148,"threads_started":1}
I20260806 12:28:48.922667 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.915361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:46823 (local address 127.0.0.1:39448)
0806 12:28:48.916028 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:48.916041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:48.916132 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:48.916688 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:48.916692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:48.916711 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:48.916880 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.916887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.918288 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.918291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.919823 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:48.919835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.921168 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.921171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:48.921263 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.921288 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 12:28:48.921355 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 12:28:48.921424 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":162,"thread_start_us":85,"threads_started":1}
I20260806 12:28:48.921861 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:48.915643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.6.152.129:46823)
0806 12:28:48.916057 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:48.916075 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:48.916101 (+ 26us) server_negotiation.cc:407] Connection header received
0806 12:28:48.916258 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:48.916262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:48.916406 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:48.916536 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:48.917035 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.918126 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.919963 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:48.920821 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:48.920935 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:48.921097 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:48.921590 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 12:28:48.921695 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
I20260806 12:28:48.924377 7769 tablet_service.cc:1467] Processing CreateTablet for tablet 8adb7921f86a477eb0a5177134e6510d (DEFAULT_TABLE table=test-table [id=8310858451de4bbc9438a20bbe218604]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:48.924388 7626 tablet_service.cc:1467] Processing CreateTablet for tablet 8adb7921f86a477eb0a5177134e6510d (DEFAULT_TABLE table=test-table [id=8310858451de4bbc9438a20bbe218604]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:48.925336 7489 tablet_service.cc:1467] Processing CreateTablet for tablet 8adb7921f86a477eb0a5177134e6510d (DEFAULT_TABLE table=test-table [id=8310858451de4bbc9438a20bbe218604]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:48.925657 7769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8adb7921f86a477eb0a5177134e6510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:48.926054 7489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8adb7921f86a477eb0a5177134e6510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:48.927920 7626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8adb7921f86a477eb0a5177134e6510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:48.932217 7857 tablet_bootstrap.cc:492] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: Bootstrap starting.
I20260806 12:28:48.933724 7858 tablet_bootstrap.cc:492] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380: Bootstrap starting.
I20260806 12:28:48.934782 7859 tablet_bootstrap.cc:492] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: Bootstrap starting.
I20260806 12:28:48.935379 7857 tablet_bootstrap.cc:654] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:48.936269 7858 tablet_bootstrap.cc:654] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:48.936478 7857 log.cc:824] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:48.937565 7858 log.cc:824] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:48.937614 7859 tablet_bootstrap.cc:654] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:48.938347 7859 log.cc:824] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:48.938695 7857 tablet_bootstrap.cc:492] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: No bootstrap required, opened a new log
I20260806 12:28:48.938879 7857 ts_tablet_manager.cc:1397] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 12:28:48.939366 7858 tablet_bootstrap.cc:492] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380: No bootstrap required, opened a new log
I20260806 12:28:48.939548 7858 ts_tablet_manager.cc:1397] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:28:48.940393 7859 tablet_bootstrap.cc:492] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: No bootstrap required, opened a new log
I20260806 12:28:48.940572 7859 ts_tablet_manager.cc:1397] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:28:48.942337 7858 raft_consensus.cc:348] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:48.942610 7858 raft_consensus.cc:374] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:48.942663 7858 raft_consensus.cc:729] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d878161b7d4233aef424f40537e380, State: Initialized, Role: FOLLOWER
I20260806 12:28:48.942726 7857 raft_consensus.cc:348] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:48.943059 7857 raft_consensus.cc:374] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:48.943136 7857 raft_consensus.cc:729] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a414a54fae419baed3283f35bf9fcb, State: Initialized, Role: FOLLOWER
I20260806 12:28:48.943107 7858 consensus_queue.cc:260] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:48.943567 7857 consensus_queue.cc:260] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:48.943981 7556 heartbeater.cc:503] Master 127.6.152.190:42789 was elected leader, sending a full tablet report...
I20260806 12:28:48.944205 7858 ts_tablet_manager.cc:1428] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260806 12:28:48.944303 7859 raft_consensus.cc:348] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:48.944567 7859 raft_consensus.cc:374] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:48.944630 7859 raft_consensus.cc:729] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e24b050cb14dc4afc1153ec9a78f76, State: Initialized, Role: FOLLOWER
I20260806 12:28:48.944957 7857 ts_tablet_manager.cc:1428] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260806 12:28:48.945035 7859 consensus_queue.cc:260] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:48.945276 7839 heartbeater.cc:503] Master 127.6.152.190:42789 was elected leader, sending a full tablet report...
I20260806 12:28:48.946543 7696 heartbeater.cc:503] Master 127.6.152.190:42789 was elected leader, sending a full tablet report...
I20260806 12:28:48.946808 7859 ts_tablet_manager.cc:1428] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:28:49.078564 7864 raft_consensus.cc:482] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:49.078823 7864 raft_consensus.cc:504] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:49.080050 7864 leader_election.cc:288] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0e24b050cb14dc4afc1153ec9a78f76 (127.6.152.130:45063), 21d878161b7d4233aef424f40537e380 (127.6.152.129:46823)
I20260806 12:28:49.084216 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.080429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:50425 (local address 127.6.152.130:45063)
0806 12:28:49.080692 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:49.080697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:49.080714 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:28:49.080774 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:49.080780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:49.080863 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:49.080943 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:49.081254 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.081902 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.083089 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.083619 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.083711 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.083752 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:49.084013 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 12:28:49.084083 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260806 12:28:49.084208 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.080263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:45063 (local address 127.6.152.131:50425)
0806 12:28:49.080431 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:49.080445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:49.080686 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:49.080955 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:49.080958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:49.080976 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:49.081129 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:49.081136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.082018 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.082021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:49.082958 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:49.082967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.083731 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.083735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:49.083855 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.083883 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 12:28:49.083972 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 12:28:49.084068 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 12:28:49.084507 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.080554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:46823 (local address 127.6.152.131:41903)
0806 12:28:49.080919 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:49.080936 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:49.081040 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:49.081423 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:49.081426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:49.081446 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:49.081560 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:49.081566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.082419 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.082421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:49.083317 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:49.083328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.084059 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.084062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:49.084158 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.084181 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:28:49.084266 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 12:28:49.084340 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":159,"threads_started":1}
I20260806 12:28:49.084501 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.081010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:41903 (local address 127.6.152.129:46823)
0806 12:28:49.081185 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:49.081192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:49.081209 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:28:49.081268 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:49.081276 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:49.081320 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:49.081380 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:49.081685 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.082303 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.083439 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.083942 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.084052 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.084097 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:49.084298 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 12:28:49.084379 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 12:28:49.085088 7646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8adb7921f86a477eb0a5177134e6510d" candidate_uuid: "80a414a54fae419baed3283f35bf9fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" is_pre_election: true
I20260806 12:28:49.085220 7509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8adb7921f86a477eb0a5177134e6510d" candidate_uuid: "80a414a54fae419baed3283f35bf9fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21d878161b7d4233aef424f40537e380" is_pre_election: true
I20260806 12:28:49.085520 7509 raft_consensus.cc:2454] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80a414a54fae419baed3283f35bf9fcb in term 0.
I20260806 12:28:49.085522 7646 raft_consensus.cc:2454] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80a414a54fae419baed3283f35bf9fcb in term 0.
I20260806 12:28:49.086112 7719 leader_election.cc:302] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21d878161b7d4233aef424f40537e380, 80a414a54fae419baed3283f35bf9fcb; no voters:
I20260806 12:28:49.086349 7864 raft_consensus.cc:2781] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:49.086447 7864 raft_consensus.cc:482] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:49.086493 7864 raft_consensus.cc:3037] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:49.087610 7864 raft_consensus.cc:504] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:49.088028 7864 leader_election.cc:288] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [CANDIDATE]: Term 1 election: Requested vote from peers f0e24b050cb14dc4afc1153ec9a78f76 (127.6.152.130:45063), 21d878161b7d4233aef424f40537e380 (127.6.152.129:46823)
I20260806 12:28:49.088382 7646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8adb7921f86a477eb0a5177134e6510d" candidate_uuid: "80a414a54fae419baed3283f35bf9fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e24b050cb14dc4afc1153ec9a78f76"
I20260806 12:28:49.088416 7509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8adb7921f86a477eb0a5177134e6510d" candidate_uuid: "80a414a54fae419baed3283f35bf9fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21d878161b7d4233aef424f40537e380"
I20260806 12:28:49.088490 7646 raft_consensus.cc:3037] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:49.088593 7509 raft_consensus.cc:3037] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:49.089557 7646 raft_consensus.cc:2454] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a414a54fae419baed3283f35bf9fcb in term 1.
I20260806 12:28:49.089700 7509 raft_consensus.cc:2454] T 8adb7921f86a477eb0a5177134e6510d P 21d878161b7d4233aef424f40537e380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a414a54fae419baed3283f35bf9fcb in term 1.
I20260806 12:28:49.089876 7719 leader_election.cc:302] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80a414a54fae419baed3283f35bf9fcb, f0e24b050cb14dc4afc1153ec9a78f76; no voters:
I20260806 12:28:49.090065 7864 raft_consensus.cc:2781] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:49.090281 7864 raft_consensus.cc:686] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [term 1 LEADER]: Becoming Leader. State: Replica: 80a414a54fae419baed3283f35bf9fcb, State: Running, Role: LEADER
I20260806 12:28:49.090690 7864 consensus_queue.cc:237] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } }
I20260806 12:28:49.094134 7355 catalog_manager.cc:5495] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80a414a54fae419baed3283f35bf9fcb (127.6.152.131). New cstate: current_term: 1 leader_uuid: "80a414a54fae419baed3283f35bf9fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e24b050cb14dc4afc1153ec9a78f76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21d878161b7d4233aef424f40537e380" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 46823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80a414a54fae419baed3283f35bf9fcb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 43655 } health_report { overall_health: HEALTHY } } }
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:28:49.109604 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.100687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.6.152.129:46823)
0806 12:28:49.100846 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:49.100851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:49.100988 (+ 137us) server_negotiation.cc:407] Connection header received
0806 12:28:49.101098 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:49.101100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:49.101137 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:49.101204 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:49.101855 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.102340 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.103178 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.103388 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.103469 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.106084 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:49.106095 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:49.106107 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:49.106129 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:49.107811 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:49.108269 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:49.108274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:49.108275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:49.108360 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:49.108663 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:49.108667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:49.108668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:49.108914 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:49.109139 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:49.109305 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 12:28:49.109424 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 12:28:49.114910 7840 tablet.cc:2369] T 8adb7921f86a477eb0a5177134e6510d P 80a414a54fae419baed3283f35bf9fcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:28:49.140388 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.132006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.6.152.129:46823)
0806 12:28:49.132145 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:49.132150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:49.132356 (+ 206us) server_negotiation.cc:407] Connection header received
0806 12:28:49.132535 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:49.132540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:49.132580 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:49.132668 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:49.133340 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.133893 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.134763 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.134993 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.135107 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.137388 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:49.137391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:49.137393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:49.137417 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:49.138601 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:49.139035 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:49.139039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:49.139041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:49.139197 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:49.139504 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:49.139508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:49.139510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:49.139674 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:49.139820 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:49.140106 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 12:28:49.140224 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 12:28:49.145987 7697 tablet.cc:2369] T 8adb7921f86a477eb0a5177134e6510d P f0e24b050cb14dc4afc1153ec9a78f76: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:28:49.170549 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:49.161628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39482 (local address 127.6.152.129:46823)
0806 12:28:49.161803 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:49.161808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:49.161940 (+ 132us) server_negotiation.cc:407] Connection header received
0806 12:28:49.162109 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:49.162113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:49.162151 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:49.162236 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:49.162885 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.163450 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.164309 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:49.164524 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:49.164631 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:49.167329 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:49.167332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:49.167334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:49.167356 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:49.168817 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:49.169303 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:49.169306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:49.169308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:49.169394 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:49.169641 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:49.169644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:49.169645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:49.169791 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:49.169886 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:49.170156 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 12:28:49.170317 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 12:28:49.172155 7467 tablet_service.cc:2908] Rejecting scan request for tablet 8adb7921f86a477eb0a5177134e6510d: Uninitialized: clean time has not yet been initialized
I20260806 12:28:49.174467 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7423
I20260806 12:28:49.181773 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7561
I20260806 12:28:49.190814 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7701
I20260806 12:28:49.197322 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7321
2026-08-06T12:28:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2396 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7882](info): set up 2 sockets
Aug 06 12:28:49 dist-test-slave-1xrh krb5kdc[7882](info): commencing operation
krb5kdc: starting...
W20260806 12:28:50.254653 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:28:50 dist-test-slave-1xrh krb5kdc[7882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:28:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:28:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:28:50.374732 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:40145
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:40145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:50.484000 7895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:50.484233 7895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:50.484300 7895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:50.488142 7895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:28:50.488188 7895 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 12:28:50.488205 7895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:50.488222 7895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:50.488283 7895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:28:50.488310 7895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:28:50.493422 7895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:40145
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:40145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:50.494367 7895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:50.495373 7895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:50.503963 7901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:50.503980 7903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:50.503998 7900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:50.504383 7895 server_base.cc:1034] running on GCE node
I20260806 12:28:50.504928 7895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:50.506022 7895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:50.507194 7895 hybrid_clock.cc:648] HybridClock initialized: now 1786019330507174 us; error 30 us; skew 500 ppm
Aug 06 12:28:50 dist-test-slave-1xrh krb5kdc[7882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019330, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:50.509968 7895 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:28:50.510857 7895 webserver.cc:460] Webserver started at http://127.6.152.190:33147/ using document root <none> and password file <none>
I20260806 12:28:50.511441 7895 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:50.511507 7895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:50.511677 7895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:50.513660 7895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "47f13a346c0442158c15a352026eb2ed"
format_stamp: "Formatted at 2026-08-06 12:28:50 on dist-test-slave-1xrh"
server_key: "f35655893d5bf42f8743d99101553ec6"
server_key_iv: "568cbc50a6460141728cccebcbd40a62"
server_key_version: "encryptionkey@0"
I20260806 12:28:50.514217 7895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "47f13a346c0442158c15a352026eb2ed"
format_stamp: "Formatted at 2026-08-06 12:28:50 on dist-test-slave-1xrh"
server_key: "f35655893d5bf42f8743d99101553ec6"
server_key_iv: "568cbc50a6460141728cccebcbd40a62"
server_key_version: "encryptionkey@0"
I20260806 12:28:50.517920 7895 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 12:28:50.520303 7911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:50.521605 7895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:50.521706 7895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "47f13a346c0442158c15a352026eb2ed"
format_stamp: "Formatted at 2026-08-06 12:28:50 on dist-test-slave-1xrh"
server_key: "f35655893d5bf42f8743d99101553ec6"
server_key_iv: "568cbc50a6460141728cccebcbd40a62"
server_key_version: "encryptionkey@0"
I20260806 12:28:50.521790 7895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:50.536625 7895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:50.539448 7895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:50.539604 7895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:50.548128 7895 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:40145
I20260806 12:28:50.548156 7976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:40145 every 8 connection(s)
I20260806 12:28:50.549047 7895 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:28:50.550647 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 7895
I20260806 12:28:50.550745 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:28:50.551038 6754 external_mini_cluster.cc:1428] Setting key d97c7fa31771de05ad69f3bb2b7f14ec
I20260806 12:28:50.552019 7977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:50.558527 7977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: Bootstrap starting.
Aug 06 12:28:50 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:50.561067 7977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:50.562633 7977 log.cc:824] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:50.564915 7977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: No bootstrap required, opened a new log
I20260806 12:28:50.566140 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:50.552448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35396 (local address 127.6.152.190:40145)
0806 12:28:50.552869 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:50.552888 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:50.552931 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:28:50.553768 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:50.553790 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:50.554208 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:50.554573 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:50.555265 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.556736 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.557587 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.558236 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.558398 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:50.561049 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:50.561056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:50.561071 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:50.561106 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:50.563205 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:50.563680 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:50.563685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:50.563691 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:50.563835 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:50.564089 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:50.564092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:50.564094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:50.564575 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:50.564761 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:50.564951 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 12:28:50.565239 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
I20260806 12:28:50.568404 7977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f13a346c0442158c15a352026eb2ed" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40145 } }
I20260806 12:28:50.568603 7977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:50.568637 7977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f13a346c0442158c15a352026eb2ed, State: Initialized, Role: FOLLOWER
I20260806 12:28:50.568972 7977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f13a346c0442158c15a352026eb2ed" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40145 } }
I20260806 12:28:50.569075 7977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:28:50.569119 7977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:28:50.569186 7977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:50.569944 7977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f13a346c0442158c15a352026eb2ed" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40145 } }
I20260806 12:28:50.570214 7977 leader_election.cc:302] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47f13a346c0442158c15a352026eb2ed; no voters:
I20260806 12:28:50.570442 7977 leader_election.cc:288] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:28:50.570568 7983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:50.570797 7983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [term 1 LEADER]: Becoming Leader. State: Replica: 47f13a346c0442158c15a352026eb2ed, State: Running, Role: LEADER
I20260806 12:28:50.571173 7983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f13a346c0442158c15a352026eb2ed" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40145 } }
I20260806 12:28:50.571651 7977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:28:50.572896 7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47f13a346c0442158c15a352026eb2ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f13a346c0442158c15a352026eb2ed" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40145 } } }
I20260806 12:28:50.572891 7985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47f13a346c0442158c15a352026eb2ed. Latest consensus state: current_term: 1 leader_uuid: "47f13a346c0442158c15a352026eb2ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f13a346c0442158c15a352026eb2ed" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40145 } } }
I20260806 12:28:50.573060 7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:50.573060 7985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:50.573477 7992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:28:50.576583 7992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:28:50.581868 7992 catalog_manager.cc:1347] Generated new cluster ID: a659ed34f1c144b686cc79f8376e89e2
I20260806 12:28:50.581935 7992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:28:50.610594 7992 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:28:50.611675 7992 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:28:50.622135 7992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: Generated new TSK 0
I20260806 12:28:50.622790 7992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:28:50.653378 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40145
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:50.799690 8004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:50.799926 8004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:50.799968 8004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:50.803872 8004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:50.803923 8004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:50.803961 8004 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:28:50.803993 8004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:28:50.809126 8004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:40145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:50.810091 8004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:50.811092 8004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:50.820617 8012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:50.820720 8004 server_base.cc:1034] running on GCE node
W20260806 12:28:50.820633 8009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:50.820633 8010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:50.821290 8004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:50.821899 8004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:50.823091 8004 hybrid_clock.cc:648] HybridClock initialized: now 1786019330823067 us; error 39 us; skew 500 ppm
Aug 06 12:28:50 dist-test-slave-1xrh krb5kdc[7882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019330, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:50.825891 8004 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:28:50.826629 8004 webserver.cc:460] Webserver started at http://127.6.152.129:36785/ using document root <none> and password file <none>
I20260806 12:28:50.827179 8004 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:50.827245 8004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:50.827400 8004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:50.829306 8004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "991f49ed12284a3b9d97f04c3c59d161"
format_stamp: "Formatted at 2026-08-06 12:28:50 on dist-test-slave-1xrh"
server_key: "60b77eed2b832eeabe4b376fcfcaf12e"
server_key_iv: "224aeeb101c5e44cceadb9f36792d0d6"
server_key_version: "encryptionkey@0"
I20260806 12:28:50.829856 8004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "991f49ed12284a3b9d97f04c3c59d161"
format_stamp: "Formatted at 2026-08-06 12:28:50 on dist-test-slave-1xrh"
server_key: "60b77eed2b832eeabe4b376fcfcaf12e"
server_key_iv: "224aeeb101c5e44cceadb9f36792d0d6"
server_key_version: "encryptionkey@0"
I20260806 12:28:50.833334 8004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:28:50.835489 8019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:50.836537 8004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:50.836649 8004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "991f49ed12284a3b9d97f04c3c59d161"
format_stamp: "Formatted at 2026-08-06 12:28:50 on dist-test-slave-1xrh"
server_key: "60b77eed2b832eeabe4b376fcfcaf12e"
server_key_iv: "224aeeb101c5e44cceadb9f36792d0d6"
server_key_version: "encryptionkey@0"
I20260806 12:28:50.836742 8004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:50.856036 8004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:50.858696 8004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:50.858897 8004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:50.860313 8004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:50.860363 8004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:50.860426 8004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:50.860466 8004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:28:50 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019330, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:50.870472 8004 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:41419
I20260806 12:28:50.870569 8143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:41419 every 8 connection(s)
I20260806 12:28:50.871471 8004 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:28:50.874912 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:50.861319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49767 (local address 127.6.152.190:40145)
0806 12:28:50.861473 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:50.861479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:50.862280 (+ 801us) server_negotiation.cc:407] Connection header received
0806 12:28:50.863400 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:50.863406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:50.863468 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:50.863539 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:50.865281 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.866120 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.867805 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.868062 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.868178 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:50.871113 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:50.871139 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:50.871142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:50.871170 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:50.872871 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:50.873509 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:50.873511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:50.873512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:50.873624 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:50.873953 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:50.873956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:50.873958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:50.874154 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:50.874244 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:50.874671 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 12:28:50.874786 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 12:28:50.875635 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:50.861661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.6.152.129:49767)
0806 12:28:50.862112 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:50.862160 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:50.863130 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:50.863979 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:50.863986 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:50.864459 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:50.865140 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:50.865160 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.866243 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.866246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:50.867685 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:50.867699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.868212 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.868217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:50.868420 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:50.869052 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:50.869074 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:50.870939 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:50.873006 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:50.873011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:50.873034 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:50.873382 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:50.873715 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:50.873717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:50.873719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:50.873848 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:50.874246 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:50.874250 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:50.874442 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 12:28:50.874590 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 12:28:50.874805 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":147,"threads_started":1}
I20260806 12:28:50.876899 8144 heartbeater.cc:347] Connected to a master server at 127.6.152.190:40145
I20260806 12:28:50.877061 8144 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:50.877614 8144 heartbeater.cc:511] Master 127.6.152.190:40145 requested a full tablet report, sending...
I20260806 12:28:50.879180 7929 ts_manager.cc:194] Registered new tserver with Master: 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419)
I20260806 12:28:50.880301 7929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:49767
I20260806 12:28:50.880509 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8004
I20260806 12:28:50.880612 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:28:50.880923 6754 external_mini_cluster.cc:1428] Setting key 4a9d54c701a904c094611d45e5e0db04
I20260806 12:28:50.886894 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:50.879224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:59523 (local address 127.6.152.190:40145)
0806 12:28:50.879327 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:50.879333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:50.879507 (+ 174us) server_negotiation.cc:407] Connection header received
0806 12:28:50.879617 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:50.879620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:50.879662 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:50.879736 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:50.880156 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.880631 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.881719 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.882028 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.882144 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:50.883474 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:50.883477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:50.883479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:50.883499 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:50.885269 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:50.885786 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:50.885788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:50.885789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:50.885860 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:50.886122 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:50.886125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:50.886126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:50.886284 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:50.886388 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:50.886652 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 12:28:50.886756 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:28:50.886904 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:50.879153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.6.152.129:59523)
0806 12:28:50.879431 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:50.879442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:50.879544 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:50.879753 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:50.879755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:50.879943 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:50.880030 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:50.880035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.880768 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.880771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:50.881591 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:50.881602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:50.882179 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:50.882184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:50.882276 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:50.882764 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:50.882783 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:50.883334 (+ 551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:50.885398 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:50.885404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:50.885413 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:50.885690 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:50.885950 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:50.885954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:50.885957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:50.886043 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:50.886384 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:50.886387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:50.886478 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 12:28:50.886584 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:28:50.886716 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:28:50.909567 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40145
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:51.016911 8155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:51.017238 8155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:51.017324 8155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:51.021276 8155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:51.021347 8155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:51.021412 8155 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:28:51.021485 8155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:28:51.026623 8155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:40145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:51.027640 8155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:51.028604 8155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:51.037686 8160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:51.037794 8161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:51.037717 8163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:51.037822 8155 server_base.cc:1034] running on GCE node
I20260806 12:28:51.038424 8155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:51.039013 8155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:51.040194 8155 hybrid_clock.cc:648] HybridClock initialized: now 1786019331040180 us; error 27 us; skew 500 ppm
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:51.042999 8155 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:28:51.043895 8155 webserver.cc:460] Webserver started at http://127.6.152.130:35733/ using document root <none> and password file <none>
I20260806 12:28:51.044433 8155 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:51.044523 8155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:51.044739 8155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:51.046876 8155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "ece66b1583824f9394600953bcd3db95"
format_stamp: "Formatted at 2026-08-06 12:28:51 on dist-test-slave-1xrh"
server_key: "c0e329a061c1fc576ccb58adeaeed6f9"
server_key_iv: "c850dc29320068a802ebffbc686d82fb"
server_key_version: "encryptionkey@0"
I20260806 12:28:51.047564 8155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "ece66b1583824f9394600953bcd3db95"
format_stamp: "Formatted at 2026-08-06 12:28:51 on dist-test-slave-1xrh"
server_key: "c0e329a061c1fc576ccb58adeaeed6f9"
server_key_iv: "c850dc29320068a802ebffbc686d82fb"
server_key_version: "encryptionkey@0"
I20260806 12:28:51.051693 8155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 12:28:51.054037 8173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:51.055274 8155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:51.055423 8155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "ece66b1583824f9394600953bcd3db95"
format_stamp: "Formatted at 2026-08-06 12:28:51 on dist-test-slave-1xrh"
server_key: "c0e329a061c1fc576ccb58adeaeed6f9"
server_key_iv: "c850dc29320068a802ebffbc686d82fb"
server_key_version: "encryptionkey@0"
I20260806 12:28:51.055510 8155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:51.088136 8155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:51.091003 8155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:51.091233 8155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:51.092656 8155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:51.092730 8155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:51.092803 8155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:51.092844 8155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:51.104219 8155 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:40193
I20260806 12:28:51.104290 8294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:40193 every 8 connection(s)
I20260806 12:28:51.105247 8155 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:28:51.106527 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.093521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:48213 (local address 127.6.152.190:40145)
0806 12:28:51.093643 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.093646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.094764 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 12:28:51.095894 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.095898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.095985 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.096053 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.097460 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.098109 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.099448 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.099672 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.099784 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.102698 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.102702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.102709 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.102735 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.104443 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.104920 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.104925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.104927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.105020 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.105411 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.105414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.105415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.105611 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.105722 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.106290 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.106400 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:28:51.107321 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.094078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.6.152.130:48213)
0806 12:28:51.094618 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.094659 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.095622 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.096431 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.096437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.096754 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:51.097300 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.097322 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.098245 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.098248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.099318 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.099329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.099839 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.099843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.100042 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.100704 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:51.100728 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:51.102524 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:51.104549 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.104554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.104564 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.104805 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.105124 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.105127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.105129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.105267 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.105778 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:51.105783 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:51.106056 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.106213 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.106436 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":184,"threads_started":1}
I20260806 12:28:51.108597 8295 heartbeater.cc:347] Connected to a master server at 127.6.152.190:40145
I20260806 12:28:51.108744 8295 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:51.109211 8295 heartbeater.cc:511] Master 127.6.152.190:40145 requested a full tablet report, sending...
I20260806 12:28:51.110180 7925 ts_manager.cc:194] Registered new tserver with Master: ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193)
I20260806 12:28:51.110913 7925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:48213
I20260806 12:28:51.115391 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8155
I20260806 12:28:51.115525 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:28:51.115854 6754 external_mini_cluster.cc:1428] Setting key eac9038a4bebd67d46e17287c0c4fcd3
I20260806 12:28:51.117607 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.110609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:53225 (local address 127.6.152.190:40145)
0806 12:28:51.110752 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.110758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.110916 (+ 158us) server_negotiation.cc:407] Connection header received
0806 12:28:51.111060 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.111063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.111112 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.111242 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.111698 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.112185 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.113004 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.113232 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.113332 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.114346 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.114349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.114352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.114375 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.115795 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.116361 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.116364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.116365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.116443 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.116753 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.116758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.116761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.116990 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.117101 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.117391 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.117479 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 12:28:51.117648 8301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.110550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.6.152.130:53225)
0806 12:28:51.110834 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.110849 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.110955 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.111259 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.111262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.111491 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:51.111588 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.111594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.112310 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.112313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.112869 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.112877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.113346 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.113350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.113451 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.113910 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:51.113922 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:51.114202 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:51.115922 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.115927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.115930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.116125 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.116552 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.116555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.116557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.116642 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.117110 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:51.117112 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:51.117220 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.117322 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.117446 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:28:51.142457 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40145
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:51.250497 8304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:51.250758 8304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:51.250804 8304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:51.254609 8304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:51.254654 8304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:51.254703 8304 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:28:51.254752 8304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:28:51.262666 8304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:40145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:51.264065 8304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:51.265278 8304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:51.277496 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:51.277503 8310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:51.277648 8312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:51.277746 8304 server_base.cc:1034] running on GCE node
I20260806 12:28:51.278198 8304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:51.278824 8304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:51.279991 8304 hybrid_clock.cc:648] HybridClock initialized: now 1786019331279964 us; error 47 us; skew 500 ppm
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:51.283339 8304 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:28:51.284464 8304 webserver.cc:460] Webserver started at http://127.6.152.131:33909/ using document root <none> and password file <none>
I20260806 12:28:51.285187 8304 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:51.285269 8304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:51.285495 8304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:51.288425 8304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "adf1526510854d7c8e608620e99d5738"
format_stamp: "Formatted at 2026-08-06 12:28:51 on dist-test-slave-1xrh"
server_key: "e578cfc3498e1041beb809fdd00f108e"
server_key_iv: "41c4a7a3bb50e158f87017c72060077c"
server_key_version: "encryptionkey@0"
I20260806 12:28:51.289187 8304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "adf1526510854d7c8e608620e99d5738"
format_stamp: "Formatted at 2026-08-06 12:28:51 on dist-test-slave-1xrh"
server_key: "e578cfc3498e1041beb809fdd00f108e"
server_key_iv: "41c4a7a3bb50e158f87017c72060077c"
server_key_version: "encryptionkey@0"
I20260806 12:28:51.294425 8304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 12:28:51.297959 8320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:51.299543 8304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 12:28:51.299669 8304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "adf1526510854d7c8e608620e99d5738"
format_stamp: "Formatted at 2026-08-06 12:28:51 on dist-test-slave-1xrh"
server_key: "e578cfc3498e1041beb809fdd00f108e"
server_key_iv: "41c4a7a3bb50e158f87017c72060077c"
server_key_version: "encryptionkey@0"
I20260806 12:28:51.299822 8304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:51.318313 8304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:51.321187 8304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:51.321367 8304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:51.322732 8304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:51.322780 8304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:51.322842 8304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:51.322878 8304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:51.334434 8304 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:36571
I20260806 12:28:51.334575 8438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:36571 every 8 connection(s)
I20260806 12:28:51.335714 8304 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:28:51.338065 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8304
I20260806 12:28:51.338195 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:28:51.338356 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.323620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:56959 (local address 127.6.152.190:40145)
0806 12:28:51.323739 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.323742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.324789 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 12:28:51.325789 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.325792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.325856 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.325933 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.327694 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.328565 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.330262 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.330628 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.330747 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.334672 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.334676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.334679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.334705 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.336446 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.336958 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.336961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.336962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.337044 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.337286 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.337289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.337290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.337477 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.337564 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.338049 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.338181 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:28:51.338502 6754 external_mini_cluster.cc:1428] Setting key cf52e5e963a43a6b949223d7fa253aa4
I20260806 12:28:51.339279 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.324171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.6.152.131:56959)
0806 12:28:51.324633 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.324672 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.325516 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.326342 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.326353 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.326858 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:51.327533 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.327554 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.328686 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.328690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.330141 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.330154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.330844 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.330850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.331101 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.332650 (+ 1549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:51.332674 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:51.334493 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:51.336590 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.336595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.336609 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.336861 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.337108 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.337110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.337111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.337216 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.337574 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:51.337579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:51.337776 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.338019 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.338355 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260806 12:28:51.340507 8439 heartbeater.cc:347] Connected to a master server at 127.6.152.190:40145
I20260806 12:28:51.340641 8439 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:51.341145 8439 heartbeater.cc:511] Master 127.6.152.190:40145 requested a full tablet report, sending...
I20260806 12:28:51.342191 7929 ts_manager.cc:194] Registered new tserver with Master: adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571)
I20260806 12:28:51.342851 7929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:56959
I20260806 12:28:51.349319 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.342587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:40095 (local address 127.6.152.190:40145)
0806 12:28:51.342708 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.342712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.342960 (+ 248us) server_negotiation.cc:407] Connection header received
0806 12:28:51.343042 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.343045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.343105 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.343193 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.343769 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.344305 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.345162 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.345400 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.345504 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.346417 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.346421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.346423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.346447 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.347678 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.348154 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.348158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.348160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.348239 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.348528 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.348533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.348535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.348704 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.348809 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.349064 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.349168 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:28:51.349330 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.342547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.6.152.131:40095)
0806 12:28:51.342849 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.342865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.342968 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.343403 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.343405 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.343562 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:51.343654 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.343659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.344451 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.344454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.345029 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.345045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.345509 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.345513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.345609 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.345991 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:51.346001 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:51.346284 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:51.347840 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.347843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.347845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.348047 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.348345 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.348348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.348350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.348438 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.348808 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:51.348811 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:51.348903 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.349007 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.349156 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":123,"threads_started":1}
I20260806 12:28:51.352335 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:51.382915 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.370276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.6.152.190:40145)
0806 12:28:51.370414 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.370418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.370429 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:28:51.370464 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.370467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.370501 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.370583 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.371336 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.371892 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.372802 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.373027 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.373139 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.380019 (+ 6880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.380022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.380024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.380054 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.381251 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.381665 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.381671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.381676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.381758 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.382027 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.382038 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.382040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.382219 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.382363 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.382626 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.382744 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 12:28:51.385728 7929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:28:51.388798 7929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:51.402390 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.396777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41419 (local address 127.0.0.1:50830)
0806 12:28:51.397571 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.397620 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.397758 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.398367 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.398373 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.398399 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.398537 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.398546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.399783 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.399789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.401005 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.401014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.401782 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.401786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.401950 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.402006 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.402125 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.402247 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":42,"thread_start_us":71,"threads_started":1}
I20260806 12:28:51.402390 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.396922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50830 (local address 127.6.152.129:41419)
0806 12:28:51.397245 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.397252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.397903 (+ 651us) server_negotiation.cc:407] Connection header received
0806 12:28:51.397991 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.397998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.398210 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.398329 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.398679 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.399676 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.401133 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.401656 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.401779 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.401945 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.402159 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.402249 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":67,"threads_started":1}
I20260806 12:28:51.403892 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.396947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40193 (local address 127.0.0.1:39594)
0806 12:28:51.397753 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.397769 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.397983 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.398920 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.398923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.398940 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.399059 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.399066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.400607 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.400612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.401470 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.401480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.403461 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.403465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.403572 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.403600 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.403659 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.403735 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":23,"thread_start_us":102,"threads_started":1}
I20260806 12:28:51.404526 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.397930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.6.152.131:36571)
0806 12:28:51.398232 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.398249 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.398642 (+ 393us) server_negotiation.cc:407] Connection header received
0806 12:28:51.398764 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.398768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.398933 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.399089 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.399372 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.400499 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.402815 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.403640 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.403714 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.403872 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.404287 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.404383 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":118,"threads_started":1}
I20260806 12:28:51.404690 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.397405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.6.152.130:40193)
0806 12:28:51.398475 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.398487 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.398510 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:28:51.398585 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.398593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.398776 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.398935 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.399217 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.400419 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.401604 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.402394 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.404255 (+ 1861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.404406 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.404451 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.404543 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":113,"threads_started":1}
I20260806 12:28:51.404976 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.397468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36571 (local address 127.0.0.1:53754)
0806 12:28:51.398522 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.398544 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.398651 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.399095 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.399098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.399116 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.399260 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.399265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.400622 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.400624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.402677 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.402690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.403981 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.403985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.404122 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.404148 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.404767 (+ 619us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.404856 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":51,"thread_start_us":89,"threads_started":1}
I20260806 12:28:51.405514 8070 tablet_service.cc:1467] Processing CreateTablet for tablet 8c074331b64b4b8d9c27e0d240257d3a (DEFAULT_TABLE table=test-table [id=f182720b5a05438e888f8b141c25584b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:51.406654 8070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c074331b64b4b8d9c27e0d240257d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.407269 8224 tablet_service.cc:1467] Processing CreateTablet for tablet 8c074331b64b4b8d9c27e0d240257d3a (DEFAULT_TABLE table=test-table [id=f182720b5a05438e888f8b141c25584b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:51.407887 8357 tablet_service.cc:1467] Processing CreateTablet for tablet 8c074331b64b4b8d9c27e0d240257d3a (DEFAULT_TABLE table=test-table [id=f182720b5a05438e888f8b141c25584b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:51.408286 8224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c074331b64b4b8d9c27e0d240257d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.408746 8357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c074331b64b4b8d9c27e0d240257d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.414214 8462 tablet_bootstrap.cc:492] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Bootstrap starting.
I20260806 12:28:51.415094 8463 tablet_bootstrap.cc:492] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Bootstrap starting.
I20260806 12:28:51.416361 8464 tablet_bootstrap.cc:492] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Bootstrap starting.
I20260806 12:28:51.416815 8462 tablet_bootstrap.cc:654] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.417033 8463 tablet_bootstrap.cc:654] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.417747 8463 log.cc:824] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:51.417747 8462 log.cc:824] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:51.418219 8464 tablet_bootstrap.cc:654] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.418941 8464 log.cc:824] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:51.419703 8463 tablet_bootstrap.cc:492] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: No bootstrap required, opened a new log
I20260806 12:28:51.419891 8463 ts_tablet_manager.cc:1397] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:28:51.420408 8462 tablet_bootstrap.cc:492] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: No bootstrap required, opened a new log
I20260806 12:28:51.420589 8462 ts_tablet_manager.cc:1397] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 12:28:51.420923 8464 tablet_bootstrap.cc:492] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: No bootstrap required, opened a new log
I20260806 12:28:51.421103 8464 ts_tablet_manager.cc:1397] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:28:51.423780 8464 raft_consensus.cc:348] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.424021 8464 raft_consensus.cc:374] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.424079 8464 raft_consensus.cc:729] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece66b1583824f9394600953bcd3db95, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.423946 8463 raft_consensus.cc:348] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.424196 8463 raft_consensus.cc:374] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.424249 8463 raft_consensus.cc:729] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf1526510854d7c8e608620e99d5738, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.424556 8464 consensus_queue.cc:260] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.424638 8463 consensus_queue.cc:260] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.425186 8295 heartbeater.cc:503] Master 127.6.152.190:40145 was elected leader, sending a full tablet report...
I20260806 12:28:51.425086 8462 raft_consensus.cc:348] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.425319 8462 raft_consensus.cc:374] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.425381 8462 raft_consensus.cc:729] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991f49ed12284a3b9d97f04c3c59d161, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.425820 8463 ts_tablet_manager.cc:1428] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:28:51.425784 8462 consensus_queue.cc:260] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.426220 8439 heartbeater.cc:503] Master 127.6.152.190:40145 was elected leader, sending a full tablet report...
I20260806 12:28:51.426659 8464 ts_tablet_manager.cc:1428] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260806 12:28:51.426659 8462 ts_tablet_manager.cc:1428] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 12:28:51.426963 8144 heartbeater.cc:503] Master 127.6.152.190:40145 was elected leader, sending a full tablet report...
I20260806 12:28:51.430471 8468 raft_consensus.cc:482] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:51.430682 8468 raft_consensus.cc:504] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.431659 8468 leader_election.cc:288] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419), adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571)
I20260806 12:28:51.436471 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.431863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41419 (local address 127.6.152.130:53845)
0806 12:28:51.431968 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.431981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.432056 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.432791 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.432794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.432811 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.432931 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.432939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.434021 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.434024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.435477 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.435488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.436149 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.436152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.436240 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.436260 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.436315 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.436369 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":25}
I20260806 12:28:51.436538 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.432507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:53845 (local address 127.6.152.129:41419)
0806 12:28:51.432614 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.432617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.432627 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:28:51.432661 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.432663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.432694 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.432779 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.433150 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.433915 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.435594 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.436033 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.436147 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.436198 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.436347 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.436414 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:28:51.436975 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c074331b64b4b8d9c27e0d240257d3a" candidate_uuid: "ece66b1583824f9394600953bcd3db95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991f49ed12284a3b9d97f04c3c59d161" is_pre_election: true
I20260806 12:28:51.437232 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.432110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36571 (local address 127.6.152.130:47637)
0806 12:28:51.432391 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.432405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.432502 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.433072 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.433075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.433109 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.433233 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.433240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.434150 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.434152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.435894 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.435901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.436850 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.436853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.436923 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.436940 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.437000 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.437069 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260806 12:28:51.437235 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.432186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:47637 (local address 127.6.152.131:36571)
0806 12:28:51.432298 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.432301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.432481 (+ 180us) server_negotiation.cc:407] Connection header received
0806 12:28:51.432900 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.432903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.432945 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.433062 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.433352 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.434062 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.436002 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.436580 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.436710 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.436774 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.437046 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.437136 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:28:51.437251 8094 raft_consensus.cc:2454] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece66b1583824f9394600953bcd3db95 in term 0.
I20260806 12:28:51.437804 8374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c074331b64b4b8d9c27e0d240257d3a" candidate_uuid: "ece66b1583824f9394600953bcd3db95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf1526510854d7c8e608620e99d5738" is_pre_election: true
I20260806 12:28:51.437878 8174 leader_election.cc:302] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991f49ed12284a3b9d97f04c3c59d161, ece66b1583824f9394600953bcd3db95; no voters:
I20260806 12:28:51.438045 8374 raft_consensus.cc:2454] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ece66b1583824f9394600953bcd3db95 in term 0.
I20260806 12:28:51.438102 8468 raft_consensus.cc:2781] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:51.438186 8468 raft_consensus.cc:482] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:51.438241 8468 raft_consensus.cc:3037] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.439252 8468 raft_consensus.cc:504] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.439563 8468 leader_election.cc:288] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [CANDIDATE]: Term 1 election: Requested vote from peers 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419), adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571)
I20260806 12:28:51.439919 8094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c074331b64b4b8d9c27e0d240257d3a" candidate_uuid: "ece66b1583824f9394600953bcd3db95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991f49ed12284a3b9d97f04c3c59d161"
I20260806 12:28:51.439980 8374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c074331b64b4b8d9c27e0d240257d3a" candidate_uuid: "ece66b1583824f9394600953bcd3db95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf1526510854d7c8e608620e99d5738"
I20260806 12:28:51.440057 8094 raft_consensus.cc:3037] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.440115 8374 raft_consensus.cc:3037] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.441088 8094 raft_consensus.cc:2454] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece66b1583824f9394600953bcd3db95 in term 1.
I20260806 12:28:51.441301 8374 raft_consensus.cc:2454] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ece66b1583824f9394600953bcd3db95 in term 1.
I20260806 12:28:51.441507 8174 leader_election.cc:302] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991f49ed12284a3b9d97f04c3c59d161, ece66b1583824f9394600953bcd3db95; no voters:
I20260806 12:28:51.441697 8468 raft_consensus.cc:2781] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:51.441956 8468 raft_consensus.cc:686] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 1 LEADER]: Becoming Leader. State: Replica: ece66b1583824f9394600953bcd3db95, State: Running, Role: LEADER
I20260806 12:28:51.442271 8468 consensus_queue.cc:237] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.445068 7929 catalog_manager.cc:5495] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 reported cstate change: term changed from 0 to 1, leader changed from <none> to ece66b1583824f9394600953bcd3db95 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "ece66b1583824f9394600953bcd3db95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } health_report { overall_health: UNKNOWN } } }
I20260806 12:28:51.465879 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.462591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.6.152.130:40193)
0806 12:28:51.462725 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.462732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.462773 (+ 41us) server_negotiation.cc:407] Connection header received
0806 12:28:51.462959 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.462962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.463007 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.463085 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:51.463311 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.463768 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.464605 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.464811 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.464884 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.465190 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.465575 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.465662 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.465738 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.465783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:28:51.471928 8094 raft_consensus.cc:1261] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Refusing update from remote peer ece66b1583824f9394600953bcd3db95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:51.472569 8472 consensus_queue.cc:1035] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:51.473830 8374 raft_consensus.cc:1261] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Refusing update from remote peer ece66b1583824f9394600953bcd3db95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:51.474431 8472 consensus_queue.cc:1035] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:51.483224 8477 mvcc.cc:205] Tried to move back new op lower bound from 7315535181701201920 to 7315535181589811200. Current Snapshot: MvccSnapshot[applied={T|T < 7315535181701201920}]
I20260806 12:28:51.484216 8479 mvcc.cc:205] Tried to move back new op lower bound from 7315535181701201920 to 7315535181589811200. Current Snapshot: MvccSnapshot[applied={T|T < 7315535181701201920}]
I20260806 12:28:51.492357 7929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:51.492569 7929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:51.495005 7929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: Sending DeleteTablet for 3 replicas of tablet 8c074331b64b4b8d9c27e0d240257d3a
I20260806 12:28:51.495930 8357 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c074331b64b4b8d9c27e0d240257d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:51 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:53754
I20260806 12:28:51.495995 8224 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c074331b64b4b8d9c27e0d240257d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:51 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:39594
I20260806 12:28:51.496071 8070 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c074331b64b4b8d9c27e0d240257d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:51 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:50830
I20260806 12:28:51.496817 7929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:28:51.497212 7929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:51.498834 8487 tablet_replica.cc:330] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: stopping tablet replica
I20260806 12:28:51.499202 8486 tablet_replica.cc:330] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: stopping tablet replica
I20260806 12:28:51.499230 8485 tablet_replica.cc:330] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: stopping tablet replica
I20260806 12:28:51.499619 8487 raft_consensus.cc:2229] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:51.499956 8486 raft_consensus.cc:2229] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:28:51.500308 8487 raft_consensus.cc:2258] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:51.500738 8486 raft_consensus.cc:2258] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:51.502607 8070 tablet_service.cc:1467] Processing CreateTablet for tablet 0051e1734faa49078db46364c1e45cc9 (DEFAULT_TABLE table=test-table [id=fce6ab3a2d3d47ccb74e2e43a6fed150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:51.502630 8224 tablet_service.cc:1467] Processing CreateTablet for tablet 0051e1734faa49078db46364c1e45cc9 (DEFAULT_TABLE table=test-table [id=fce6ab3a2d3d47ccb74e2e43a6fed150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:51.502804 8357 tablet_service.cc:1467] Processing CreateTablet for tablet 0051e1734faa49078db46364c1e45cc9 (DEFAULT_TABLE table=test-table [id=fce6ab3a2d3d47ccb74e2e43a6fed150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:51.502882 8070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0051e1734faa49078db46364c1e45cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.502882 8224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0051e1734faa49078db46364c1e45cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.503067 8357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0051e1734faa49078db46364c1e45cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.505882 8462 tablet_bootstrap.cc:492] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Bootstrap starting.
I20260806 12:28:51.505882 8464 tablet_bootstrap.cc:492] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Bootstrap starting.
I20260806 12:28:51.506672 8463 tablet_bootstrap.cc:492] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Bootstrap starting.
I20260806 12:28:51.507490 8464 tablet_bootstrap.cc:654] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.507669 8462 tablet_bootstrap.cc:654] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.508716 8463 tablet_bootstrap.cc:654] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.509668 8485 raft_consensus.cc:2229] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:51.510020 8485 raft_consensus.cc:2258] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:51.510397 8487 ts_tablet_manager.cc:1905] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:51.510985 8464 tablet_bootstrap.cc:492] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: No bootstrap required, opened a new log
I20260806 12:28:51.511102 8464 ts_tablet_manager.cc:1397] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 12:28:51.511775 8485 ts_tablet_manager.cc:1905] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:51.512013 8464 raft_consensus.cc:348] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.512156 8464 raft_consensus.cc:374] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.512224 8464 raft_consensus.cc:729] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece66b1583824f9394600953bcd3db95, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.512429 8464 consensus_queue.cc:260] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.512753 8464 ts_tablet_manager.cc:1428] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 12:28:51.513760 8486 ts_tablet_manager.cc:1905] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:51.513950 8487 ts_tablet_manager.cc:1918] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:28:51.514048 8487 log.cc:1196] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/8c074331b64b4b8d9c27e0d240257d3a
I20260806 12:28:51.515065 8487 ts_tablet_manager.cc:1939] T 8c074331b64b4b8d9c27e0d240257d3a P 991f49ed12284a3b9d97f04c3c59d161: Deleting consensus metadata
I20260806 12:28:51.516558 7912 catalog_manager.cc:4841] TS 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419): tablet 8c074331b64b4b8d9c27e0d240257d3a (table test-table [id=f182720b5a05438e888f8b141c25584b]) successfully deleted
I20260806 12:28:51.516803 8463 tablet_bootstrap.cc:492] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: No bootstrap required, opened a new log
I20260806 12:28:51.516876 8463 ts_tablet_manager.cc:1397] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20260806 12:28:51.517513 8462 tablet_bootstrap.cc:492] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: No bootstrap required, opened a new log
I20260806 12:28:51.517588 8462 ts_tablet_manager.cc:1397] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260806 12:28:51.517650 8463 raft_consensus.cc:348] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.517765 8463 raft_consensus.cc:374] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.517802 8463 raft_consensus.cc:729] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf1526510854d7c8e608620e99d5738, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.518015 8463 consensus_queue.cc:260] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.518078 8462 raft_consensus.cc:348] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.518219 8462 raft_consensus.cc:374] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.518260 8462 raft_consensus.cc:729] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991f49ed12284a3b9d97f04c3c59d161, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.518354 8486 ts_tablet_manager.cc:1918] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:28:51.518445 8486 log.cc:1196] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/8c074331b64b4b8d9c27e0d240257d3a
I20260806 12:28:51.518384 8462 consensus_queue.cc:260] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.518688 8462 ts_tablet_manager.cc:1428] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:28:51.518760 8486 ts_tablet_manager.cc:1939] T 8c074331b64b4b8d9c27e0d240257d3a P ece66b1583824f9394600953bcd3db95: Deleting consensus metadata
I20260806 12:28:51.519946 7914 catalog_manager.cc:4841] TS ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193): tablet 8c074331b64b4b8d9c27e0d240257d3a (table test-table [id=f182720b5a05438e888f8b141c25584b]) successfully deleted
I20260806 12:28:51.520470 8463 ts_tablet_manager.cc:1428] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260806 12:28:51.522501 8485 ts_tablet_manager.cc:1918] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:28:51.522578 8485 log.cc:1196] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/8c074331b64b4b8d9c27e0d240257d3a
I20260806 12:28:51.522841 8485 ts_tablet_manager.cc:1939] T 8c074331b64b4b8d9c27e0d240257d3a P adf1526510854d7c8e608620e99d5738: Deleting consensus metadata
I20260806 12:28:51.523923 7912 catalog_manager.cc:4841] TS adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571): tablet 8c074331b64b4b8d9c27e0d240257d3a (table test-table [id=f182720b5a05438e888f8b141c25584b]) successfully deleted
W20260806 12:28:51.586814 8440 tablet.cc:2369] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:28:51.606518 8296 tablet.cc:2369] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:28:51.620888 8469 raft_consensus.cc:482] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:51.621057 8469 raft_consensus.cc:504] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.622403 8469 leader_election.cc:288] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419), ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193)
W20260806 12:28:51.623234 8145 tablet.cc:2369] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:28:51.627830 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.622623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41419 (local address 127.6.152.131:57505)
0806 12:28:51.622795 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.622813 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.622916 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.623705 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.623708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.623728 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.623906 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.623914 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.624927 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.624932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.626243 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.626255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.627398 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.627405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.627515 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.627538 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.627616 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.627693 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:28:51.627830 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.622901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:57505 (local address 127.6.152.129:41419)
0806 12:28:51.623097 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.623102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.623117 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:28:51.623192 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.623195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.623244 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.623320 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.624106 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.624769 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.626417 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.627251 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.627416 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.627503 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.627646 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.627722 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 12:28:51.628204 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.623571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58185 (local address 127.6.152.130:40193)
0806 12:28:51.623718 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.623723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.623738 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:28:51.623800 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.623802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.623840 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.623945 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.624464 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.625388 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.626888 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.627352 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.627481 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.627533 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.627947 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.628039 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:28:51.628667 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0051e1734faa49078db46364c1e45cc9" candidate_uuid: "adf1526510854d7c8e608620e99d5738" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991f49ed12284a3b9d97f04c3c59d161" is_pre_election: true
I20260806 12:28:51.628806 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.622623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40193 (local address 127.6.152.131:58185)
0806 12:28:51.623109 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.623124 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.623269 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.623960 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.623963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.623982 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.624303 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.624311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.625557 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.625561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.626774 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.626783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.627618 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.627622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.627766 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.627790 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.628490 (+ 700us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.628609 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":140,"thread_start_us":83,"threads_started":1}
I20260806 12:28:51.628832 8092 raft_consensus.cc:2454] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf1526510854d7c8e608620e99d5738 in term 0.
I20260806 12:28:51.629596 8321 leader_election.cc:302] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991f49ed12284a3b9d97f04c3c59d161, adf1526510854d7c8e608620e99d5738; no voters:
I20260806 12:28:51.629756 8245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0051e1734faa49078db46364c1e45cc9" candidate_uuid: "adf1526510854d7c8e608620e99d5738" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece66b1583824f9394600953bcd3db95" is_pre_election: true
I20260806 12:28:51.629881 8469 raft_consensus.cc:2781] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:51.629946 8469 raft_consensus.cc:482] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:51.629988 8469 raft_consensus.cc:3037] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.629987 8245 raft_consensus.cc:2454] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf1526510854d7c8e608620e99d5738 in term 0.
I20260806 12:28:51.631738 8469 raft_consensus.cc:504] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.632180 8469 leader_election.cc:288] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [CANDIDATE]: Term 1 election: Requested vote from peers 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419), ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193)
I20260806 12:28:51.632539 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0051e1734faa49078db46364c1e45cc9" candidate_uuid: "adf1526510854d7c8e608620e99d5738" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991f49ed12284a3b9d97f04c3c59d161"
I20260806 12:28:51.632652 8092 raft_consensus.cc:3037] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.632805 8245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0051e1734faa49078db46364c1e45cc9" candidate_uuid: "adf1526510854d7c8e608620e99d5738" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece66b1583824f9394600953bcd3db95"
I20260806 12:28:51.632918 8245 raft_consensus.cc:3037] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.633816 8092 raft_consensus.cc:2454] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf1526510854d7c8e608620e99d5738 in term 1.
I20260806 12:28:51.634074 8321 leader_election.cc:302] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991f49ed12284a3b9d97f04c3c59d161, adf1526510854d7c8e608620e99d5738; no voters:
I20260806 12:28:51.634239 8469 raft_consensus.cc:2781] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:51.634410 8469 raft_consensus.cc:686] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 1 LEADER]: Becoming Leader. State: Replica: adf1526510854d7c8e608620e99d5738, State: Running, Role: LEADER
I20260806 12:28:51.634642 8469 consensus_queue.cc:237] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.634763 8245 raft_consensus.cc:2454] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf1526510854d7c8e608620e99d5738 in term 1.
I20260806 12:28:51.637236 7925 catalog_manager.cc:5495] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 reported cstate change: term changed from 0 to 1, leader changed from <none> to adf1526510854d7c8e608620e99d5738 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "adf1526510854d7c8e608620e99d5738" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } health_report { overall_health: HEALTHY } } }
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019330, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:51.703469 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.692799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40145 (local address 127.0.0.1:35414)
0806 12:28:51.693190 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.693203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.693422 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.693707 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.693713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.694504 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:51.694704 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.694714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.695548 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.695555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.696805 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.696820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.697313 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.697319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.697453 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.698157 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:51.698177 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:51.699884 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:51.701629 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.701639 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.701657 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.702055 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.702345 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:51.702351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:51.702357 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:51.702482 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:51.702860 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:51.702870 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:51.703022 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.703198 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.703326 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":125,"threads_started":1}
I20260806 12:28:51.703510 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.692902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.6.152.190:40145)
0806 12:28:51.693068 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.693074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.693381 (+ 307us) server_negotiation.cc:407] Connection header received
0806 12:28:51.693534 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.693536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.693580 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.693643 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.694852 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.695436 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.696970 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.697184 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.697266 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.700042 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.700046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.700048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.700073 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.701468 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.702172 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.702175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.702177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.702250 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.702589 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.702591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.702592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.702756 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.702827 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.703243 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.703399 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":58}
I20260806 12:28:51.707787 7925 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:35414:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 12:28:51.708626 7925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:51.713428 8224 tablet_service.cc:1467] Processing CreateTablet for tablet 618e6cc3ea7f4202ac1a2db265cd5742 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55b450cb9fc643e5a10135cc288e9373]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:28:51.713886 8224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618e6cc3ea7f4202ac1a2db265cd5742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.713428 8070 tablet_service.cc:1467] Processing CreateTablet for tablet 618e6cc3ea7f4202ac1a2db265cd5742 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55b450cb9fc643e5a10135cc288e9373]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:28:51.715934 8070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618e6cc3ea7f4202ac1a2db265cd5742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.716764 8464 tablet_bootstrap.cc:492] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: Bootstrap starting.
I20260806 12:28:51.717247 8357 tablet_service.cc:1467] Processing CreateTablet for tablet 618e6cc3ea7f4202ac1a2db265cd5742 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55b450cb9fc643e5a10135cc288e9373]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:28:51.717667 8357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618e6cc3ea7f4202ac1a2db265cd5742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:51.717867 8464 tablet_bootstrap.cc:654] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.718935 8464 tablet_bootstrap.cc:492] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: No bootstrap required, opened a new log
I20260806 12:28:51.719053 8464 ts_tablet_manager.cc:1397] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 12:28:51.720279 8464 raft_consensus.cc:348] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.720397 8464 raft_consensus.cc:374] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.720446 8464 raft_consensus.cc:729] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece66b1583824f9394600953bcd3db95, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.720496 8463 tablet_bootstrap.cc:492] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: Bootstrap starting.
I20260806 12:28:51.720580 8464 consensus_queue.cc:260] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.720857 8464 ts_tablet_manager.cc:1428] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:28:51.720860 8468 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.721112 8468 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.722062 8463 tablet_bootstrap.cc:654] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.724373 8463 tablet_bootstrap.cc:492] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: No bootstrap required, opened a new log
I20260806 12:28:51.724470 8463 ts_tablet_manager.cc:1397] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 12:28:51.724774 8462 tablet_bootstrap.cc:492] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: Bootstrap starting.
I20260806 12:28:51.725061 8463 raft_consensus.cc:348] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.725179 8463 raft_consensus.cc:374] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.725225 8463 raft_consensus.cc:729] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf1526510854d7c8e608620e99d5738, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.725371 8463 consensus_queue.cc:260] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.725621 8469 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.725737 8463 ts_tablet_manager.cc:1428] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:28:51.725821 8469 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.726207 8462 tablet_bootstrap.cc:654] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:51.728288 8462 tablet_bootstrap.cc:492] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: No bootstrap required, opened a new log
I20260806 12:28:51.728390 8462 ts_tablet_manager.cc:1397] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 12:28:51.729121 8462 raft_consensus.cc:348] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.729218 8462 raft_consensus.cc:374] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:51.729241 8462 raft_consensus.cc:729] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991f49ed12284a3b9d97f04c3c59d161, State: Initialized, Role: FOLLOWER
I20260806 12:28:51.729337 8462 consensus_queue.cc:260] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.729606 8462 ts_tablet_manager.cc:1428] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:28:51.729571 8470 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.729773 8470 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.845046 8470 raft_consensus.cc:482] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:51.845184 8470 raft_consensus.cc:504] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.846323 8470 leader_election.cc:288] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193), adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571)
I20260806 12:28:51.850980 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.846690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:39787 (local address 127.6.152.131:36571)
0806 12:28:51.846860 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.846864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.847646 (+ 782us) server_negotiation.cc:407] Connection header received
0806 12:28:51.847793 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.847797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.847842 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.847922 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.848210 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.848709 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.849962 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.850349 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.850476 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.850527 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.850759 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.850849 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:28:51.851004 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.846647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49849 (local address 127.6.152.130:40193)
0806 12:28:51.846807 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.846812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.847510 (+ 698us) server_negotiation.cc:407] Connection header received
0806 12:28:51.847704 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.847707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.847751 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.847829 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:51.848274 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.848759 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.849958 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.850352 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.850474 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.850527 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.850787 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.850882 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:28:51.851063 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.846568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36571 (local address 127.6.152.129:39787)
0806 12:28:51.847367 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.847379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.847684 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.847935 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.847938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.847955 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.848104 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.848110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.848815 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.848819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.849821 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.849831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.850471 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.850475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.850637 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.850656 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.850741 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.850812 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":63,"thread_start_us":271,"threads_started":1}
I20260806 12:28:51.851480 8503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.846530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40193 (local address 127.6.152.129:49849)
0806 12:28:51.847165 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.847182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.847561 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.847852 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.847856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.847873 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:51.848151 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.848158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.848870 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.848875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.849829 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.849839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.850474 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.850478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.850662 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.850680 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.850765 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.850852 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":42,"thread_start_us":626,"threads_started":1}
I20260806 12:28:51.851605 8374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e6cc3ea7f4202ac1a2db265cd5742" candidate_uuid: "991f49ed12284a3b9d97f04c3c59d161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf1526510854d7c8e608620e99d5738" is_pre_election: true
I20260806 12:28:51.851768 8374 raft_consensus.cc:2454] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991f49ed12284a3b9d97f04c3c59d161 in term 0.
I20260806 12:28:51.851989 8245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e6cc3ea7f4202ac1a2db265cd5742" candidate_uuid: "991f49ed12284a3b9d97f04c3c59d161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece66b1583824f9394600953bcd3db95" is_pre_election: true
I20260806 12:28:51.852152 8245 raft_consensus.cc:2454] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991f49ed12284a3b9d97f04c3c59d161 in term 0.
I20260806 12:28:51.852248 8020 leader_election.cc:302] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991f49ed12284a3b9d97f04c3c59d161, adf1526510854d7c8e608620e99d5738; no voters:
I20260806 12:28:51.852521 8470 raft_consensus.cc:2781] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:51.852635 8470 raft_consensus.cc:482] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:51.852694 8470 raft_consensus.cc:3037] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.853920 8470 raft_consensus.cc:504] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.854297 8470 leader_election.cc:288] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [CANDIDATE]: Term 1 election: Requested vote from peers ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193), adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571)
I20260806 12:28:51.854588 8245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e6cc3ea7f4202ac1a2db265cd5742" candidate_uuid: "991f49ed12284a3b9d97f04c3c59d161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ece66b1583824f9394600953bcd3db95"
I20260806 12:28:51.854637 8374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618e6cc3ea7f4202ac1a2db265cd5742" candidate_uuid: "991f49ed12284a3b9d97f04c3c59d161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf1526510854d7c8e608620e99d5738"
I20260806 12:28:51.854725 8245 raft_consensus.cc:3037] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.854753 8374 raft_consensus.cc:3037] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:51.855868 8245 raft_consensus.cc:2454] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991f49ed12284a3b9d97f04c3c59d161 in term 1.
I20260806 12:28:51.856002 8374 raft_consensus.cc:2454] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991f49ed12284a3b9d97f04c3c59d161 in term 1.
I20260806 12:28:51.856200 8022 leader_election.cc:302] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991f49ed12284a3b9d97f04c3c59d161, ece66b1583824f9394600953bcd3db95; no voters:
I20260806 12:28:51.856382 8470 raft_consensus.cc:2781] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:51.856596 8470 raft_consensus.cc:686] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 LEADER]: Becoming Leader. State: Replica: 991f49ed12284a3b9d97f04c3c59d161, State: Running, Role: LEADER
I20260806 12:28:51.856856 8470 consensus_queue.cc:237] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } }
I20260806 12:28:51.858314 8505 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: TxnStatusTablet state changed. Reason: New leader 991f49ed12284a3b9d97f04c3c59d161. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.858445 8505 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:28:51.858884 8507 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 12:28:51.859155 8507 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 12:28:51.859804 7925 catalog_manager.cc:5495] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 reported cstate change: term changed from 0 to 1, leader changed from <none> to 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } health_report { overall_health: UNKNOWN } } }
I20260806 12:28:51.873872 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.868927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41419 (local address 127.0.0.1:50834)
0806 12:28:51.869258 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.869274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.869368 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.869697 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.869700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.869706 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:51.869802 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.869807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.870944 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.870948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.871941 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.871953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.872590 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.872594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.872685 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.872739 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.873553 (+ 814us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.873569 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.873650 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.873737 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260806 12:28:51.874095 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.869030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50834 (local address 127.6.152.129:41419)
0806 12:28:51.869171 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.869176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.869333 (+ 157us) server_negotiation.cc:407] Connection header received
0806 12:28:51.869481 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.869484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.869549 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.869667 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:51.869906 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.870812 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.872080 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.872457 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.872581 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.872917 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.873458 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.873572 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.873669 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.873725 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:28:51.877602 8374 raft_consensus.cc:1261] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Refusing update from remote peer 991f49ed12284a3b9d97f04c3c59d161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:51.877755 8245 raft_consensus.cc:1261] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Refusing update from remote peer 991f49ed12284a3b9d97f04c3c59d161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:51.878189 8470 consensus_queue.cc:1035] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:51.878351 8505 consensus_queue.cc:1035] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:51.879741 8469 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: TxnStatusTablet state changed. Reason: New leader 991f49ed12284a3b9d97f04c3c59d161. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.879873 8469 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.881373 8468 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: TxnStatusTablet state changed. Reason: New leader 991f49ed12284a3b9d97f04c3c59d161. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.881494 8468 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.886859 8505 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.886968 8505 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:28:51.887975 8476 mvcc.cc:205] Tried to move back new op lower bound from 7315535183364583424 to 7315535183288487936. Current Snapshot: MvccSnapshot[applied={T|T < 7315535183364583424}]
I20260806 12:28:51.888633 8470 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.888737 8470 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P 991f49ed12284a3b9d97f04c3c59d161: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:28:51.889046 8468 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.889145 8468 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P ece66b1583824f9394600953bcd3db95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.890525 8469 tablet_replica.cc:439] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "991f49ed12284a3b9d97f04c3c59d161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 } } peers { permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 } } peers { permanent_uuid: "adf1526510854d7c8e608620e99d5738" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36571 } } }
I20260806 12:28:51.890636 8469 tablet_replica.cc:442] T 618e6cc3ea7f4202ac1a2db265cd5742 P adf1526510854d7c8e608620e99d5738: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:51.896927 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.892433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.6.152.131:36571)
0806 12:28:51.892540 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.892544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.892748 (+ 204us) server_negotiation.cc:407] Connection header received
0806 12:28:51.892872 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.892876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.892946 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.893050 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:51.893294 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.894062 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.895227 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.895572 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.895687 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.895952 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.896469 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.896590 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.896731 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.896801 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 12:28:51.907624 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.904012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:41451 (local address 127.6.152.129:41419)
0806 12:28:51.904137 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.904141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.904436 (+ 295us) server_negotiation.cc:407] Connection header received
0806 12:28:51.904585 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.904588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.904635 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.904709 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:51.904973 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.905611 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.906538 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.906753 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.906852 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.907239 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.907306 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.907379 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.907468 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.907510 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:28:51.907802 8515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.903941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41419 (local address 127.6.152.131:41451)
0806 12:28:51.904351 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.904365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.904476 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.904730 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.904733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.904740 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:51.904870 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.904876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.905712 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.905717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.906374 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.906383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.906910 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.906915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.907028 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.907078 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.907395 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.907401 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.907459 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.907542 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":151,"threads_started":1}
I20260806 12:28:51.914294 8245 raft_consensus.cc:1261] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Refusing update from remote peer adf1526510854d7c8e608620e99d5738: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:51.914654 8092 raft_consensus.cc:1261] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Refusing update from remote peer adf1526510854d7c8e608620e99d5738: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:51.914891 8492 consensus_queue.cc:1035] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ece66b1583824f9394600953bcd3db95" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:51.915181 8469 consensus_queue.cc:1035] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991f49ed12284a3b9d97f04c3c59d161" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:51.939024 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.934292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36571 (local address 127.6.152.129:46351)
0806 12:28:51.934880 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:51.934894 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:51.935015 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:51.935450 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:51.935453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:51.935462 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:51.935659 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.935666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.936522 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.936526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.937317 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:51.937327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.937814 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.937818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:51.937967 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.938012 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.938477 (+ 465us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.938486 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 12:28:51.938578 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 12:28:51.938673 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":231,"threads_started":1}
I20260806 12:28:51.939157 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.934284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:46351 (local address 127.6.152.131:36571)
0806 12:28:51.935252 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.935255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.935266 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:28:51.935300 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.935302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.935338 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.935430 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:51.935802 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.936412 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.937472 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.937704 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.937811 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.938169 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:51.938249 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:51.938472 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.938910 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.938970 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895}
I20260806 12:28:51.969092 7925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:51.969300 7925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:51.971766 7925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47f13a346c0442158c15a352026eb2ed: Sending DeleteTablet for 3 replicas of tablet 0051e1734faa49078db46364c1e45cc9
I20260806 12:28:51.972468 8070 tablet_service.cc:1514] Processing DeleteTablet for tablet 0051e1734faa49078db46364c1e45cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:51 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:50830
I20260806 12:28:51.972626 8224 tablet_service.cc:1514] Processing DeleteTablet for tablet 0051e1734faa49078db46364c1e45cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:51 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:39594
I20260806 12:28:51.972811 8486 tablet_replica.cc:330] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: stopping tablet replica
I20260806 12:28:51.972841 8487 tablet_replica.cc:330] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: stopping tablet replica
I20260806 12:28:51.972865 8357 tablet_service.cc:1514] Processing DeleteTablet for tablet 0051e1734faa49078db46364c1e45cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:51 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:53754
I20260806 12:28:51.972942 8487 raft_consensus.cc:2229] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:51.972935 8486 raft_consensus.cc:2229] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:51.973043 8487 raft_consensus.cc:2258] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:51.973043 8486 raft_consensus.cc:2258] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:51.973203 8485 tablet_replica.cc:330] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: stopping tablet replica
I20260806 12:28:51.973307 8485 raft_consensus.cc:2229] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:28:51.973657 8485 raft_consensus.cc:2258] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:51.974488 8485 ts_tablet_manager.cc:1905] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:51.974545 8486 ts_tablet_manager.cc:1905] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:51.975045 8487 ts_tablet_manager.cc:1905] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:51.977706 8486 ts_tablet_manager.cc:1918] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:51.977798 8486 log.cc:1196] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/0051e1734faa49078db46364c1e45cc9
I20260806 12:28:51.978091 8486 ts_tablet_manager.cc:1939] T 0051e1734faa49078db46364c1e45cc9 P ece66b1583824f9394600953bcd3db95: Deleting consensus metadata
I20260806 12:28:51.978232 8485 ts_tablet_manager.cc:1918] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:51.978289 8487 ts_tablet_manager.cc:1918] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:51.978379 8487 log.cc:1196] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/0051e1734faa49078db46364c1e45cc9
I20260806 12:28:51.978483 8485 log.cc:1196] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/0051e1734faa49078db46364c1e45cc9
I20260806 12:28:51.978662 8487 ts_tablet_manager.cc:1939] T 0051e1734faa49078db46364c1e45cc9 P 991f49ed12284a3b9d97f04c3c59d161: Deleting consensus metadata
I20260806 12:28:51.978875 8485 ts_tablet_manager.cc:1939] T 0051e1734faa49078db46364c1e45cc9 P adf1526510854d7c8e608620e99d5738: Deleting consensus metadata
I20260806 12:28:51.978859 7914 catalog_manager.cc:4841] TS ece66b1583824f9394600953bcd3db95 (127.6.152.130:40193): tablet 0051e1734faa49078db46364c1e45cc9 (table test-table [id=fce6ab3a2d3d47ccb74e2e43a6fed150]) successfully deleted
I20260806 12:28:51.979393 7912 catalog_manager.cc:4841] TS 991f49ed12284a3b9d97f04c3c59d161 (127.6.152.129:41419): tablet 0051e1734faa49078db46364c1e45cc9 (table test-table [id=fce6ab3a2d3d47ccb74e2e43a6fed150]) successfully deleted
I20260806 12:28:51.979552 7912 catalog_manager.cc:4841] TS adf1526510854d7c8e608620e99d5738 (127.6.152.131:36571): tablet 0051e1734faa49078db46364c1e45cc9 (table test-table [id=fce6ab3a2d3d47ccb74e2e43a6fed150]) successfully deleted
Aug 06 12:28:51 dist-test-slave-1xrh krb5kdc[7882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:28:51.992364 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.980069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50838 (local address 127.6.152.129:41419)
0806 12:28:51.980197 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.980202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.980617 (+ 415us) server_negotiation.cc:407] Connection header received
0806 12:28:51.980742 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.980745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.980794 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.980889 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.981846 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.982696 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.984043 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.984392 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.984502 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:51.988466 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:51.988478 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:51.988494 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:51.988520 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:51.990449 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.990970 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.990976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.990978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.991066 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:51.991371 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:51.991374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:51.991375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:51.991662 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:51.991848 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:51.992077 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 12:28:51.992206 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 12:28:51.993304 8138 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50838
I20260806 12:28:52.004736 7980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:51.995110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.6.152.190:40145)
0806 12:28:51.995248 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:51.995252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:51.995870 (+ 618us) server_negotiation.cc:407] Connection header received
0806 12:28:51.995989 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:51.995992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:51.996045 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:51.996126 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:51.996953 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.997845 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.998901 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:51.999310 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:51.999411 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:52.000942 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:52.000946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:52.000949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:52.000977 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:52.002845 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:52.003393 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:52.003397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:52.003399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:52.003486 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:52.003824 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:52.003827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:52.003829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:52.004059 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:52.004207 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:52.004460 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 12:28:52.004576 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 12:28:52.005390 7925 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35418
I20260806 12:28:52.007301 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8004
I20260806 12:28:52.016830 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8155
I20260806 12:28:52.025128 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8304
I20260806 12:28:52.032303 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 7895
2026-08-06T12:28:52Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2837 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:28:52 dist-test-slave-1xrh krb5kdc[8530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:28:52 dist-test-slave-1xrh krb5kdc[8530](info): set up 2 sockets
Aug 06 12:28:52 dist-test-slave-1xrh krb5kdc[8530](info): commencing operation
krb5kdc: starting...
W20260806 12:28:53.163662 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:28:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:28:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:28:53.282195 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35345
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:53.393774 8545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:53.394035 8545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:53.394091 8545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:53.398175 8545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:28:53.398227 8545 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 12:28:53.398245 8545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:53.398263 8545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:53.398279 8545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:28:53.398329 8545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:28:53.403505 8545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35345
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:53.404495 8545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:53.405465 8545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:53.413806 8551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:53.413830 8553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:53.413981 8545 server_base.cc:1034] running on GCE node
W20260806 12:28:53.413995 8550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:53.414585 8545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:53.415673 8545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:53.416867 8545 hybrid_clock.cc:648] HybridClock initialized: now 1786019333416843 us; error 35 us; skew 500 ppm
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:53.419798 8545 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:28:53.420660 8545 webserver.cc:460] Webserver started at http://127.6.152.190:39559/ using document root <none> and password file <none>
I20260806 12:28:53.421152 8545 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:53.421209 8545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:53.421366 8545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:53.423385 8545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "b0a5c026859f4c089b1e327508cf79c0"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "cfb7dffbe5c233e38b5f1d4d638ba68c"
server_key_iv: "3188391b1548648b6495e798be5ba788"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.423899 8545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "b0a5c026859f4c089b1e327508cf79c0"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "cfb7dffbe5c233e38b5f1d4d638ba68c"
server_key_iv: "3188391b1548648b6495e798be5ba788"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.427609 8545 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:28:53.429939 8561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:53.431097 8545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:53.431278 8545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "b0a5c026859f4c089b1e327508cf79c0"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "cfb7dffbe5c233e38b5f1d4d638ba68c"
server_key_iv: "3188391b1548648b6495e798be5ba788"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.431381 8545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:53.451118 8545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:53.454269 8545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:53.454424 8545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:53.462513 8545 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:35345
I20260806 12:28:53.462536 8627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:35345 every 8 connection(s)
I20260806 12:28:53.463390 8545 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:28:53.466104 8628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:53.467836 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8545
I20260806 12:28:53.467933 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:28:53.468209 6754 external_mini_cluster.cc:1428] Setting key e59df5d1cfe819c9a175376749a18ca6
I20260806 12:28:53.472894 8628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Bootstrap starting.
I20260806 12:28:53.475454 8628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:53.476194 8628 log.cc:824] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:53.478008 8628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: No bootstrap required, opened a new log
I20260806 12:28:53.482292 8628 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a5c026859f4c089b1e327508cf79c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35345 } }
I20260806 12:28:53.482561 8628 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:53.482621 8628 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a5c026859f4c089b1e327508cf79c0, State: Initialized, Role: FOLLOWER
I20260806 12:28:53.483072 8628 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a5c026859f4c089b1e327508cf79c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35345 } }
I20260806 12:28:53.483247 8628 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:28:53.483312 8628 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:28:53.483402 8628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:53.483809 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.469667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47866 (local address 127.6.152.190:35345)
0806 12:28:53.470075 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:53.470093 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:53.470136 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:28:53.470972 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:53.470995 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:53.471449 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:53.471799 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:53.472474 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.473873 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.474732 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.475422 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.475590 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.477926 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:53.477932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:53.477947 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:53.477975 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:53.480846 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.481250 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.481256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.481262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.481414 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.481662 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.481666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.481667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.482160 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:53.482372 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:53.482562 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 12:28:53.482846 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
I20260806 12:28:53.484457 8628 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a5c026859f4c089b1e327508cf79c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35345 } }
I20260806 12:28:53.484787 8628 leader_election.cc:302] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0a5c026859f4c089b1e327508cf79c0; no voters:
I20260806 12:28:53.485118 8628 leader_election.cc:288] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:28:53.485217 8633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:53.485442 8633 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [term 1 LEADER]: Becoming Leader. State: Replica: b0a5c026859f4c089b1e327508cf79c0, State: Running, Role: LEADER
I20260806 12:28:53.485724 8633 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a5c026859f4c089b1e327508cf79c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35345 } }
I20260806 12:28:53.486352 8628 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:28:53.486901 8633 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0a5c026859f4c089b1e327508cf79c0. Latest consensus state: current_term: 1 leader_uuid: "b0a5c026859f4c089b1e327508cf79c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a5c026859f4c089b1e327508cf79c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35345 } } }
I20260806 12:28:53.487023 8633 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:53.487712 8635 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0a5c026859f4c089b1e327508cf79c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a5c026859f4c089b1e327508cf79c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35345 } } }
I20260806 12:28:53.487871 8635 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0 [sys.catalog]: This master's current role is: LEADER
W20260806 12:28:53.490813 8650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:28:53.490876 8650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:28:53.490959 8638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:28:53.491781 8638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:28:53.496470 8638 catalog_manager.cc:1347] Generated new cluster ID: 6f8639d801914e72bdda1fe4f46fa720
I20260806 12:28:53.496533 8638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:28:53.510133 8638 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:28:53.510985 8638 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:28:53.517105 8638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Generated new TSK 0
I20260806 12:28:53.517815 8638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:28:53.553510 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35345
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:53.664606 8655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:53.664875 8655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:53.664954 8655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:53.669014 8655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:53.669083 8655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:53.669148 8655 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 12:28:53.669188 8655 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:28:53.669251 8655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:28:53.674484 8655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:35345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:53.675539 8655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:53.676494 8655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:53.685954 8661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:53.685936 8662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:53.685928 8664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:53.686061 8655 server_base.cc:1034] running on GCE node
I20260806 12:28:53.686636 8655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:53.687326 8655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:53.688501 8655 hybrid_clock.cc:648] HybridClock initialized: now 1786019333688470 us; error 69 us; skew 500 ppm
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:53.691161 8655 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:28:53.692152 8655 webserver.cc:460] Webserver started at http://127.6.152.129:36811/ using document root <none> and password file <none>
I20260806 12:28:53.692723 8655 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:53.692850 8655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:53.693074 8655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:53.695005 8655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "bf483819a21540c1864bcef2b9769328"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "15b2ff9fe94262daa01e951fa95e1345"
server_key_iv: "508be8be4976895f0a3496b18425c457"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.695979 8655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "bf483819a21540c1864bcef2b9769328"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "15b2ff9fe94262daa01e951fa95e1345"
server_key_iv: "508be8be4976895f0a3496b18425c457"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.699904 8655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 12:28:53.702167 8671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:53.703464 8655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:53.703603 8655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "bf483819a21540c1864bcef2b9769328"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "15b2ff9fe94262daa01e951fa95e1345"
server_key_iv: "508be8be4976895f0a3496b18425c457"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.703715 8655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:53.720578 8655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:53.723380 8655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:53.723572 8655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:53.724960 8655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:53.725031 8655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:53.725095 8655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:53.725134 8655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:53.736321 8655 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:41967
I20260806 12:28:53.736387 8798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:41967 every 8 connection(s)
I20260806 12:28:53.737443 8655 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:28:53.739753 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8655
I20260806 12:28:53.739889 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:28:53.740031 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.725817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49949 (local address 127.6.152.190:35345)
0806 12:28:53.725944 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:53.725948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:53.727009 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 12:28:53.728013 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:53.728016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:53.728085 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:53.728171 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:53.729703 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.730605 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.732109 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.732563 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.732709 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.736214 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:53.736217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:53.736219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:53.736243 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:53.737703 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.738421 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.738423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.738425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.738500 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.738858 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.738861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.738863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.739046 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:53.739215 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:53.739722 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 12:28:53.739857 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:28:53.740235 6754 external_mini_cluster.cc:1428] Setting key 3f98d5b5c36848f08a34bf358374396f
I20260806 12:28:53.740823 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.726366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.6.152.129:49949)
0806 12:28:53.726857 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:53.726897 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:53.727776 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:53.728348 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:53.728357 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:53.728862 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:53.729556 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.729572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.730740 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.730744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.731948 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.731960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.732751 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.732758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.732989 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.734107 (+ 1118us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:53.734126 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:53.736006 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:53.737817 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.737822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.737835 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.738167 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.738595 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.738598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.738600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.738753 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.739203 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:53.739209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:53.739475 (+ 266us) client_negotiation.cc:769] Sending connection context
0806 12:28:53.739661 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 12:28:53.739973 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":148,"threads_started":1}
I20260806 12:28:53.742247 8799 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35345
I20260806 12:28:53.742400 8799 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:53.742959 8799 heartbeater.cc:511] Master 127.6.152.190:35345 requested a full tablet report, sending...
I20260806 12:28:53.744575 8578 ts_manager.cc:194] Registered new tserver with Master: bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967)
I20260806 12:28:53.745649 8578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:49949
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:28:53.754714 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.744402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:45801 (local address 127.6.152.190:35345)
0806 12:28:53.744508 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:53.744512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:53.747481 (+ 2969us) server_negotiation.cc:407] Connection header received
0806 12:28:53.747607 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:53.747610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:53.747656 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:53.747758 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:53.748651 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.749108 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.750365 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.750564 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.750636 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.751911 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:53.751914 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:53.751916 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:53.751938 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:53.753186 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.753568 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.753571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.753572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.753644 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.753880 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.753882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.753883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.754122 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:53.754232 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:53.754509 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 12:28:53.754608 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:28:53.754755 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.744386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.6.152.129:45801)
0806 12:28:53.747377 (+ 2991us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:53.747396 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:53.747510 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:53.747817 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:53.747823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:53.748317 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:53.748531 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.748543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.749236 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.749241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.750244 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.750258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.750683 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.750690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.750820 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.751353 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:53.751373 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:53.751788 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:53.753276 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.753278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.753279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.753450 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.753725 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.753726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.753728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.753805 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.754232 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:53.754238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:53.754335 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 12:28:53.754424 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 12:28:53.754559 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2877,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:28:53.769506 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35345
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:53.876575 8809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:53.876806 8809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:53.876883 8809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:53.881506 8809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:53.881559 8809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:53.881628 8809 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 12:28:53.881659 8809 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:28:53.881726 8809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:28:53.886832 8809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:35345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:53.887821 8809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:53.888770 8809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:53.897927 8815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:53.897960 8816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:53.897981 8809 server_base.cc:1034] running on GCE node
W20260806 12:28:53.897948 8818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:53.898497 8809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:53.899106 8809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:53.900329 8809 hybrid_clock.cc:648] HybridClock initialized: now 1786019333900300 us; error 40 us; skew 500 ppm
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:53.902969 8809 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:28:53.903833 8809 webserver.cc:460] Webserver started at http://127.6.152.130:39017/ using document root <none> and password file <none>
I20260806 12:28:53.904362 8809 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:53.904450 8809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:53.904661 8809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:53.906517 8809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "faacf54df216422eb2e530bc31140285"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "6f8f152c8a93772227a24fa5e9adbbbc"
server_key_iv: "482b16602207081a916cb90751f0556e"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.907058 8809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "faacf54df216422eb2e530bc31140285"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "6f8f152c8a93772227a24fa5e9adbbbc"
server_key_iv: "482b16602207081a916cb90751f0556e"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.911059 8809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 12:28:53.913467 8825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:53.914577 8809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 12:28:53.914709 8809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "faacf54df216422eb2e530bc31140285"
format_stamp: "Formatted at 2026-08-06 12:28:53 on dist-test-slave-1xrh"
server_key: "6f8f152c8a93772227a24fa5e9adbbbc"
server_key_iv: "482b16602207081a916cb90751f0556e"
server_key_version: "encryptionkey@0"
I20260806 12:28:53.914809 8809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:53.930855 8809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:53.933822 8809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:53.934005 8809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:53.935635 8809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:53.935700 8809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:53.935765 8809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:53.935797 8809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:28:53 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:53.946403 8809 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:44331
I20260806 12:28:53.946509 8956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:44331 every 8 connection(s)
I20260806 12:28:53.947353 8809 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:28:53.949965 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.936243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:54685 (local address 127.6.152.190:35345)
0806 12:28:53.936414 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:53.936421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:53.937743 (+ 1322us) server_negotiation.cc:407] Connection header received
0806 12:28:53.938753 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:53.938757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:53.938818 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:53.938908 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:53.940530 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.941389 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.943221 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.943437 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.943521 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.946509 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:53.946513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:53.946515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:53.946549 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:53.948266 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.948709 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.948712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.948713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.948785 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.949093 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.949095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.949096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.949256 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:53.949331 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:53.949743 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 12:28:53.949837 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:28:53.950670 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.937148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.6.152.130:54685)
0806 12:28:53.937585 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:53.937633 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:53.938549 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:53.939150 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:53.939159 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:53.939646 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:53.940343 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.940366 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.941561 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.941565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.943066 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.943080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.943624 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.943630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.943882 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.944514 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:53.944533 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:53.946304 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:53.948363 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.948367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.948376 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.948625 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.948860 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.948862 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.948863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.949013 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.949339 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:53.949344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:53.949555 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 12:28:53.949697 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 12:28:53.949918 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":127,"threads_started":1}
I20260806 12:28:53.951862 8958 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35345
I20260806 12:28:53.952008 8958 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:53.952476 8958 heartbeater.cc:511] Master 127.6.152.190:35345 requested a full tablet report, sending...
I20260806 12:28:53.953429 8578 ts_manager.cc:194] Registered new tserver with Master: faacf54df216422eb2e530bc31140285 (127.6.152.130:44331)
I20260806 12:28:53.954104 8578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:54685
I20260806 12:28:53.955343 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8809
I20260806 12:28:53.955462 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:28:53.955775 6754 external_mini_cluster.cc:1428] Setting key 45a53f06a0b95d080d88658fc3879196
I20260806 12:28:53.961287 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.953881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:33031 (local address 127.6.152.190:35345)
0806 12:28:53.954004 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:53.954010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:53.954232 (+ 222us) server_negotiation.cc:407] Connection header received
0806 12:28:53.954321 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:53.954324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:53.954365 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:53.954457 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:53.955019 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.955651 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.956674 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.956939 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.957056 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.958017 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:53.958021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:53.958023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:53.958054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:53.959581 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.959975 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.959981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.959984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.960086 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:53.960354 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:53.960359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:53.960362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:53.960666 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:53.960770 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:53.961095 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 12:28:53.961186 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:28:53.961431 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:53.953823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.6.152.130:33031)
0806 12:28:53.954131 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:53.954142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:53.954248 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:53.954474 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:53.954477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:53.954714 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:53.954898 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.954906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.955767 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.955770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.956558 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:53.956569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:53.957053 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:53.957057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:53.957151 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:53.957600 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:53.957618 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:53.957886 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:53.959690 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.959693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.959694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.959875 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.960177 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:53.960179 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:53.960180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:53.960251 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:53.960771 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:53.960777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:53.960882 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 12:28:53.961008 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 12:28:53.961174 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":18,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:28:53.983354 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35345
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:54.094440 8969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:54.094705 8969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:54.094782 8969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:54.098762 8969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:54.098850 8969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:54.098922 8969 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 12:28:54.098965 8969 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:28:54.099025 8969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:28:54.104288 8969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:35345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:54.105280 8969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:54.106241 8969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:54.115509 8974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:54.115530 8976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:54.115530 8980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:54.115738 8969 server_base.cc:1034] running on GCE node
I20260806 12:28:54.116235 8969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:54.116878 8969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:54.118060 8969 hybrid_clock.cc:648] HybridClock initialized: now 1786019334118050 us; error 28 us; skew 500 ppm
Aug 06 12:28:54 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:54.120852 8969 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:28:54.121650 8969 webserver.cc:460] Webserver started at http://127.6.152.131:42013/ using document root <none> and password file <none>
I20260806 12:28:54.122191 8969 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:54.122279 8969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:54.122495 8969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:54.124481 8969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "1fd2234e44874b80a170e69fc93e1aba"
format_stamp: "Formatted at 2026-08-06 12:28:54 on dist-test-slave-1xrh"
server_key: "4eff5b7853ef07d43a64bf5fcbbf0dc1"
server_key_iv: "64aa6707923ac0af8412dc1ade5b1bad"
server_key_version: "encryptionkey@0"
I20260806 12:28:54.125027 8969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "1fd2234e44874b80a170e69fc93e1aba"
format_stamp: "Formatted at 2026-08-06 12:28:54 on dist-test-slave-1xrh"
server_key: "4eff5b7853ef07d43a64bf5fcbbf0dc1"
server_key_iv: "64aa6707923ac0af8412dc1ade5b1bad"
server_key_version: "encryptionkey@0"
I20260806 12:28:54.128582 8969 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:28:54.130775 8988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:54.131946 8969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:54.132058 8969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "1fd2234e44874b80a170e69fc93e1aba"
format_stamp: "Formatted at 2026-08-06 12:28:54 on dist-test-slave-1xrh"
server_key: "4eff5b7853ef07d43a64bf5fcbbf0dc1"
server_key_iv: "64aa6707923ac0af8412dc1ade5b1bad"
server_key_version: "encryptionkey@0"
I20260806 12:28:54.132139 8969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:54.155026 8969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:54.158169 8969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:54.158317 8969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:54.159770 8969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:28:54.159839 8969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:54.159906 8969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:28:54.159953 8969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:28:54 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:54.171460 8969 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35505
I20260806 12:28:54.171519 9116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35505 every 8 connection(s)
I20260806 12:28:54.172564 8969 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:28:54.174011 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.160436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:40135 (local address 127.6.152.190:35345)
0806 12:28:54.160616 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.160622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.161809 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 12:28:54.163078 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.163084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.163186 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.163300 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:54.164725 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.165381 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.167089 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.167396 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.167524 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.170053 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:54.170059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:54.170065 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:54.170101 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:54.171974 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.172443 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.172447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.172449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.172558 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.172959 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.172964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.172967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.173229 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:54.173331 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.173778 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.173894 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:28:54.174784 8998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.161203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.6.152.131:40135)
0806 12:28:54.161630 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.161679 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.162781 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.163465 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.163471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.163952 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:54.164568 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.164591 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.165521 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.165524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.166966 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.166980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.167570 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.167575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.167749 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.168320 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:54.168336 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:54.169891 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:54.172089 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:54.172094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:54.172105 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:54.172341 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:54.172671 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:54.172674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:54.172676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:54.172835 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:54.173341 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:54.173347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:54.173568 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.173716 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.173945 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":118,"threads_started":1}
I20260806 12:28:54.176024 9117 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35345
I20260806 12:28:54.176159 9117 heartbeater.cc:464] Registering TS with master...
I20260806 12:28:54.176666 9117 heartbeater.cc:511] Master 127.6.152.190:35345 requested a full tablet report, sending...
I20260806 12:28:54.177752 8579 ts_manager.cc:194] Registered new tserver with Master: 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:54.178491 8579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:40135
I20260806 12:28:54.179540 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 8969
I20260806 12:28:54.179670 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:28:54.180063 6754 external_mini_cluster.cc:1428] Setting key 64d5715279c52dfe104e9575e19527eb
I20260806 12:28:54.182168 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:28:54.185670 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.178267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:57169 (local address 127.6.152.190:35345)
0806 12:28:54.178391 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.178395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.178564 (+ 169us) server_negotiation.cc:407] Connection header received
0806 12:28:54.178723 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.178725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.178779 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.178864 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:54.179378 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.180115 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.181038 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.181292 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.181395 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.182549 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:54.182552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:54.182553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:54.182575 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:54.184059 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.184526 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.184528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.184529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.184598 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.184882 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.184886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.184888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.185063 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:54.185163 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.185425 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.185533 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:28:54.185684 9124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.178212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.6.152.131:57169)
0806 12:28:54.178478 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.178494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.178595 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.178881 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.178883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.179079 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:54.179234 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.179242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.180275 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.180278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.180914 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.180923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.181409 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.181413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.181512 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.181988 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:54.182000 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:54.182430 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:54.184174 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:54.184180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:54.184184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:54.184443 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:54.184695 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:54.184700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:54.184702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:54.184791 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:54.185169 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:54.185174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:54.185263 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.185366 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.185513 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":5,"thread_start_us":102,"threads_started":1}
Aug 06 12:28:54 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:28:54 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:54.208915 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.201137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.6.152.190:35345)
0806 12:28:54.201255 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.201258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.201270 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:28:54.201302 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.201304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.201338 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.201406 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:54.202056 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.202573 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.203426 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.203635 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.203713 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.205876 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:54.205879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:54.205881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:54.205903 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:54.207430 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.207807 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.207810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.207811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.207885 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.208207 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.208210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.208211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.208383 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:54.208491 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.208699 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.208791 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:28:54.211711 8579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:28:54.214716 8579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:54.228893 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.223571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54462 (local address 127.6.152.129:41967)
0806 12:28:54.224071 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.224089 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.224112 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:28:54.224188 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.224192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.224379 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.224534 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.224791 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.225717 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.227360 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.228177 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.228295 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.228437 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.228624 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.228725 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":90,"threads_started":1}
I20260806 12:28:54.228889 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.222754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41967 (local address 127.0.0.1:54462)
0806 12:28:54.223700 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.223715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.223809 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.224508 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.224511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.224526 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.224660 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.224667 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.225817 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.225823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.227244 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.227254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.228310 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.228314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.228433 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.228475 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.228584 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.228701 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":122,"thread_start_us":86,"threads_started":1}
I20260806 12:28:54.229053 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.223370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35505 (local address 127.0.0.1:44008)
0806 12:28:54.224779 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.224792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.224898 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.225329 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.225331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.225350 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.225448 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.225453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.226762 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.226766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.227747 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.227754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.228606 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.228610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.228705 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.228727 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.228802 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.228864 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260806 12:28:54.229456 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.223956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.6.152.131:35505)
0806 12:28:54.224370 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.224381 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.224897 (+ 516us) server_negotiation.cc:407] Connection header received
0806 12:28:54.225010 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.225014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.225188 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.225334 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.225560 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.226630 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.227857 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.228469 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.228598 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.228725 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.228855 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.229321 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":123,"threads_started":1}
I20260806 12:28:54.230299 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.222652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:44331 (local address 127.0.0.1:37456)
0806 12:28:54.223276 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.223334 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.223477 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.224316 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.224322 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.224343 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.224496 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.224504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.225890 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.225893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.227346 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.227357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.229919 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.229923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.230017 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.230052 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.230131 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.230209 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":60,"thread_start_us":67,"threads_started":1}
I20260806 12:28:54.230752 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.222777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.6.152.130:44331)
0806 12:28:54.223095 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.223107 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.223730 (+ 623us) server_negotiation.cc:407] Connection header received
0806 12:28:54.223927 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.223933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.224149 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.224319 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.224610 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.225754 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.227463 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.228186 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.228295 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.228435 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.230537 (+ 2102us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.230634 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":116,"threads_started":1}
I20260806 12:28:54.231956 8725 tablet_service.cc:1467] Processing CreateTablet for tablet 6c68b0f3d11b49809bf6ae10250ef020 (DEFAULT_TABLE table=test-table [id=9cb1638f571e4468893e5187d7c72eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:54.231958 9044 tablet_service.cc:1467] Processing CreateTablet for tablet 6c68b0f3d11b49809bf6ae10250ef020 (DEFAULT_TABLE table=test-table [id=9cb1638f571e4468893e5187d7c72eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:54.233045 8725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c68b0f3d11b49809bf6ae10250ef020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.233045 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c68b0f3d11b49809bf6ae10250ef020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.232875 8882 tablet_service.cc:1467] Processing CreateTablet for tablet 6c68b0f3d11b49809bf6ae10250ef020 (DEFAULT_TABLE table=test-table [id=9cb1638f571e4468893e5187d7c72eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:54.233697 8882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c68b0f3d11b49809bf6ae10250ef020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.241425 9142 tablet_bootstrap.cc:492] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Bootstrap starting.
I20260806 12:28:54.242147 9144 tablet_bootstrap.cc:492] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Bootstrap starting.
I20260806 12:28:54.242761 9143 tablet_bootstrap.cc:492] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Bootstrap starting.
I20260806 12:28:54.243640 9142 tablet_bootstrap.cc:654] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.244185 9144 tablet_bootstrap.cc:654] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.244400 9142 log.cc:824] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:54.244791 9143 tablet_bootstrap.cc:654] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.244975 9144 log.cc:824] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:54.245533 9143 log.cc:824] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Log is configured to *not* fsync() on all Append() calls
I20260806 12:28:54.246956 9144 tablet_bootstrap.cc:492] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: No bootstrap required, opened a new log
I20260806 12:28:54.247171 9143 tablet_bootstrap.cc:492] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: No bootstrap required, opened a new log
I20260806 12:28:54.247170 9144 ts_tablet_manager.cc:1397] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 12:28:54.247193 9142 tablet_bootstrap.cc:492] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: No bootstrap required, opened a new log
I20260806 12:28:54.247349 9142 ts_tablet_manager.cc:1397] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:28:54.247368 9143 ts_tablet_manager.cc:1397] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:28:54.250015 9143 raft_consensus.cc:348] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.250450 9143 raft_consensus.cc:374] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.250519 9143 raft_consensus.cc:729] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.250989 9142 raft_consensus.cc:348] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.251024 9144 raft_consensus.cc:348] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.251276 9142 raft_consensus.cc:374] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.251289 9144 raft_consensus.cc:374] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.251339 9142 raft_consensus.cc:729] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faacf54df216422eb2e530bc31140285, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.251348 9144 raft_consensus.cc:729] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf483819a21540c1864bcef2b9769328, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.251387 9143 consensus_queue.cc:260] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.251787 9144 consensus_queue.cc:260] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.251787 9142 consensus_queue.cc:260] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.252172 9117 heartbeater.cc:503] Master 127.6.152.190:35345 was elected leader, sending a full tablet report...
I20260806 12:28:54.252490 8958 heartbeater.cc:503] Master 127.6.152.190:35345 was elected leader, sending a full tablet report...
I20260806 12:28:54.252703 9142 ts_tablet_manager.cc:1428] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 12:28:54.253708 9144 ts_tablet_manager.cc:1428] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:28:54.254855 8799 heartbeater.cc:503] Master 127.6.152.190:35345 was elected leader, sending a full tablet report...
I20260806 12:28:54.255813 9143 ts_tablet_manager.cc:1428] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260806 12:28:54.301962 9150 raft_consensus.cc:482] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:54.302186 9150 raft_consensus.cc:504] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.303459 9150 leader_election.cc:288] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faacf54df216422eb2e530bc31140285 (127.6.152.130:44331), 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:54.307345 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.303782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:56143 (local address 127.6.152.130:44331)
0806 12:28:54.303931 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.303936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.304086 (+ 150us) server_negotiation.cc:407] Connection header received
0806 12:28:54.304309 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.304313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.304369 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.304428 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.304790 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.305303 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.306385 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.306780 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.306919 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.306971 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.307158 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.307226 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:28:54.307369 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.303651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:44331 (local address 127.6.152.129:56143)
0806 12:28:54.303973 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.303989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.304170 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.304560 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.304562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.304578 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.304670 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.304674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.305426 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.305430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.306258 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.306268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.306911 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.306915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.307007 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.307036 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.307121 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.307212 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":6,"thread_start_us":122,"threads_started":1}
I20260806 12:28:54.307595 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.304077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35505 (local address 127.6.152.129:60197)
0806 12:28:54.304374 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.304388 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.304480 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.304749 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.304752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.304769 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.304887 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.304893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.305660 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.305667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.306438 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.306447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.307106 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.307109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.307243 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.307261 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.307383 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.307449 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":21,"thread_start_us":71,"threads_started":1}
I20260806 12:28:54.307585 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.304304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:60197 (local address 127.6.152.131:35505)
0806 12:28:54.304453 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.304458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.304471 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:28:54.304600 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.304603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.304652 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.304727 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.304996 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.305486 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.306570 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.306978 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.307106 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.307174 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.307404 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.307472 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:28:54.308009 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c68b0f3d11b49809bf6ae10250ef020" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285" is_pre_election: true
I20260806 12:28:54.308213 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c68b0f3d11b49809bf6ae10250ef020" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd2234e44874b80a170e69fc93e1aba" is_pre_election: true
I20260806 12:28:54.308332 8904 raft_consensus.cc:2454] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 0.
I20260806 12:28:54.308494 9068 raft_consensus.cc:2454] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 0.
I20260806 12:28:54.308851 8675 leader_election.cc:302] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf483819a21540c1864bcef2b9769328, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:54.309135 9150 raft_consensus.cc:2781] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:54.309226 9150 raft_consensus.cc:482] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:54.309276 9150 raft_consensus.cc:3037] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.310336 9150 raft_consensus.cc:504] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.310662 9150 leader_election.cc:288] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 election: Requested vote from peers faacf54df216422eb2e530bc31140285 (127.6.152.130:44331), 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:54.310992 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c68b0f3d11b49809bf6ae10250ef020" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285"
I20260806 12:28:54.311069 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c68b0f3d11b49809bf6ae10250ef020" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd2234e44874b80a170e69fc93e1aba"
I20260806 12:28:54.311123 8904 raft_consensus.cc:3037] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.311213 9068 raft_consensus.cc:3037] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.312177 9068 raft_consensus.cc:2454] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 1.
I20260806 12:28:54.312485 8675 leader_election.cc:302] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd2234e44874b80a170e69fc93e1aba, bf483819a21540c1864bcef2b9769328; no voters:
I20260806 12:28:54.312513 8904 raft_consensus.cc:2454] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 1.
I20260806 12:28:54.312682 9150 raft_consensus.cc:2781] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:54.312888 9150 raft_consensus.cc:686] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 1 LEADER]: Becoming Leader. State: Replica: bf483819a21540c1864bcef2b9769328, State: Running, Role: LEADER
I20260806 12:28:54.313251 9150 consensus_queue.cc:237] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.316301 8579 catalog_manager.cc:5495] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf483819a21540c1864bcef2b9769328 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } health_report { overall_health: UNKNOWN } } }
Aug 06 12:28:54 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019333, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:54.357023 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.348787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35345 (local address 127.0.0.1:47884)
0806 12:28:54.349095 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.349107 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.349238 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.349476 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.349478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.349899 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:28:54.349989 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.349995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.350731 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.350736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.351413 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.351426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.351847 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.351850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.351924 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.352509 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:28:54.352527 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:28:54.353927 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:28:54.355642 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:54.355650 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:54.355655 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:54.355873 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:54.356125 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:28:54.356127 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:28:54.356129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:28:54.356204 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:28:54.356526 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:28:54.356531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:28:54.356630 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.356721 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.356842 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":111,"threads_started":1}
I20260806 12:28:54.357023 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.348868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47884 (local address 127.6.152.190:35345)
0806 12:28:54.349007 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.349012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.349179 (+ 167us) server_negotiation.cc:407] Connection header received
0806 12:28:54.349341 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.349344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.349380 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.349457 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:54.350103 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.350615 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.351529 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.351742 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.351847 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.354072 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:54.354075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:54.354077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:54.354101 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:54.355551 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.355962 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.355965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.355966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.356050 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:54.356276 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:54.356279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:54.356280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:54.356442 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:54.356516 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.356777 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.356899 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":50}
I20260806 12:28:54.360808 8579 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:47884:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 12:28:54.361456 8579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:54.366444 8882 tablet_service.cc:1467] Processing CreateTablet for tablet 673914291e5b4eb6904851f060ed067e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e9e1ea91514f44eabb6176ae4b85e3da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:28:54.366487 9044 tablet_service.cc:1467] Processing CreateTablet for tablet 673914291e5b4eb6904851f060ed067e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e9e1ea91514f44eabb6176ae4b85e3da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:28:54.366269 8725 tablet_service.cc:1467] Processing CreateTablet for tablet 673914291e5b4eb6904851f060ed067e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e9e1ea91514f44eabb6176ae4b85e3da]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:28:54.367050 8882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 673914291e5b4eb6904851f060ed067e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.367045 8725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 673914291e5b4eb6904851f060ed067e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.367044 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 673914291e5b4eb6904851f060ed067e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.369444 9142 tablet_bootstrap.cc:492] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: Bootstrap starting.
I20260806 12:28:54.370146 9144 tablet_bootstrap.cc:492] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: Bootstrap starting.
I20260806 12:28:54.370319 9143 tablet_bootstrap.cc:492] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: Bootstrap starting.
I20260806 12:28:54.370518 9142 tablet_bootstrap.cc:654] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.371050 9144 tablet_bootstrap.cc:654] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.371402 9143 tablet_bootstrap.cc:654] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.372179 9142 tablet_bootstrap.cc:492] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: No bootstrap required, opened a new log
I20260806 12:28:54.372270 9142 ts_tablet_manager.cc:1397] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:28:54.372676 9142 raft_consensus.cc:348] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.372742 9144 tablet_bootstrap.cc:492] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: No bootstrap required, opened a new log
I20260806 12:28:54.372761 9142 raft_consensus.cc:374] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.372782 9142 raft_consensus.cc:729] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faacf54df216422eb2e530bc31140285, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.372799 9144 ts_tablet_manager.cc:1397] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:28:54.372915 9142 consensus_queue.cc:260] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.373173 9142 ts_tablet_manager.cc:1428] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:28:54.373195 9144 raft_consensus.cc:348] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.373200 9149 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.373306 9144 raft_consensus.cc:374] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.373338 9144 raft_consensus.cc:729] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf483819a21540c1864bcef2b9769328, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.373344 9149 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:54.373502 9144 consensus_queue.cc:260] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.373728 9150 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.373855 9150 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:54.374078 9144 ts_tablet_manager.cc:1428] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:28:54.374151 9143 tablet_bootstrap.cc:492] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: No bootstrap required, opened a new log
I20260806 12:28:54.374243 9143 ts_tablet_manager.cc:1397] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 12:28:54.374720 9143 raft_consensus.cc:348] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.374831 9143 raft_consensus.cc:374] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.374871 9143 raft_consensus.cc:729] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.375005 9143 consensus_queue.cc:260] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.375284 9143 ts_tablet_manager.cc:1428] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:28:54.375501 9148 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.375661 9148 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 12:28:54.423906 9118 tablet.cc:2369] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:28:54.441998 9150 raft_consensus.cc:482] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:54.442127 9150 raft_consensus.cc:504] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.442536 9150 leader_election.cc:288] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faacf54df216422eb2e530bc31140285 (127.6.152.130:44331), 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:54.442955 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673914291e5b4eb6904851f060ed067e" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285" is_pre_election: true
I20260806 12:28:54.443106 8904 raft_consensus.cc:2454] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 0.
I20260806 12:28:54.443032 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673914291e5b4eb6904851f060ed067e" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd2234e44874b80a170e69fc93e1aba" is_pre_election: true
I20260806 12:28:54.443250 9068 raft_consensus.cc:2454] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 0.
I20260806 12:28:54.443480 8675 leader_election.cc:302] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf483819a21540c1864bcef2b9769328, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:54.443661 9150 raft_consensus.cc:2781] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:54.443724 9150 raft_consensus.cc:482] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:54.443764 9150 raft_consensus.cc:3037] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.444821 9150 raft_consensus.cc:504] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.445228 9150 leader_election.cc:288] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 election: Requested vote from peers faacf54df216422eb2e530bc31140285 (127.6.152.130:44331), 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:54.445528 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673914291e5b4eb6904851f060ed067e" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285"
I20260806 12:28:54.445600 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673914291e5b4eb6904851f060ed067e" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd2234e44874b80a170e69fc93e1aba"
I20260806 12:28:54.445696 8904 raft_consensus.cc:3037] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.445737 9068 raft_consensus.cc:3037] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.446591 8904 raft_consensus.cc:2454] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 1.
I20260806 12:28:54.446686 9068 raft_consensus.cc:2454] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 1.
I20260806 12:28:54.446900 8675 leader_election.cc:302] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf483819a21540c1864bcef2b9769328, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:54.447110 9150 raft_consensus.cc:2781] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:54.447233 9150 raft_consensus.cc:686] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [term 1 LEADER]: Becoming Leader. State: Replica: bf483819a21540c1864bcef2b9769328, State: Running, Role: LEADER
I20260806 12:28:54.447379 9150 consensus_queue.cc:237] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.448058 9153 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: TxnStatusTablet state changed. Reason: New leader bf483819a21540c1864bcef2b9769328. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.448180 9153 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:28:54.448504 9168 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
W20260806 12:28:54.448632 8959 tablet.cc:2369] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:28:54.448762 9168 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 12:28:54.449738 8577 catalog_manager.cc:5495] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf483819a21540c1864bcef2b9769328 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } health_report { overall_health: UNKNOWN } } }
I20260806 12:28:54.474833 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.470352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41967 (local address 127.0.0.1:54478)
0806 12:28:54.470506 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.470522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.470634 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.470955 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.470957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.470964 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:54.471098 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.471104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.472181 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.472184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.473112 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.473122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.473694 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.473697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.473803 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.473853 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.474552 (+ 699us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.474563 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.474632 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.474708 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 12:28:54.474838 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.470577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.6.152.129:41967)
0806 12:28:54.470696 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.470700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.470713 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:28:54.470778 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.470781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.470837 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.470937 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:54.471222 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.472045 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.473230 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.473581 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.473682 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.474005 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.474455 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.474560 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.474656 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.474714 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:28:54.479902 8904 raft_consensus.cc:1261] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Refusing update from remote peer bf483819a21540c1864bcef2b9769328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:54.479923 9068 raft_consensus.cc:1261] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Refusing update from remote peer bf483819a21540c1864bcef2b9769328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:54.480580 9153 consensus_queue.cc:1035] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:54.480736 9150 consensus_queue.cc:1035] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:54.484262 9148 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: TxnStatusTablet state changed. Reason: New leader bf483819a21540c1864bcef2b9769328. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.484375 9148 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:54.485077 9149 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: TxnStatusTablet state changed. Reason: New leader bf483819a21540c1864bcef2b9769328. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.485191 9149 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:54.487829 9150 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.487941 9150 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:28:54.487903 9148 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.487995 9148 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P 1fd2234e44874b80a170e69fc93e1aba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:28:54.488066 9150 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.488165 9150 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P bf483819a21540c1864bcef2b9769328: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:28:54.488835 9171 mvcc.cc:205] Tried to move back new op lower bound from 7315535194023313408 to 7315535193897832448. Current Snapshot: MvccSnapshot[applied={T|T < 7315535194023313408}]
I20260806 12:28:54.489513 9149 tablet_replica.cc:439] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } } }
I20260806 12:28:54.489609 9149 tablet_replica.cc:442] T 673914291e5b4eb6904851f060ed067e P faacf54df216422eb2e530bc31140285: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 12:28:54.489800 8800 tablet.cc:2369] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:28:54.490527 9173 mvcc.cc:205] Tried to move back new op lower bound from 7315535194023313408 to 7315535193897832448. Current Snapshot: MvccSnapshot[applied={T|T < 7315535194023313408}]
I20260806 12:28:54.494118 9172 mvcc.cc:205] Tried to move back new op lower bound from 7315535194023313408 to 7315535193897832448. Current Snapshot: MvccSnapshot[applied={T|T < 7315535194023313408 or (T in {7315535194023313408})}]
I20260806 12:28:54.497720 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.494095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.6.152.129:41967)
0806 12:28:54.494193 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.494197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.494343 (+ 146us) server_negotiation.cc:407] Connection header received
0806 12:28:54.494442 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.494445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.494489 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.494566 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:54.494823 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.495574 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.496792 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.497000 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.497078 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.497326 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.497389 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.497440 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.497556 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.497595 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 12:28:54.507882 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.504448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:47631 (local address 127.6.152.129:41967)
0806 12:28:54.504553 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.504557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.504796 (+ 239us) server_negotiation.cc:407] Connection header received
0806 12:28:54.504891 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.504893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.504937 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.505007 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:54.505238 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.505955 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.506728 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.507039 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.507169 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.507448 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.507516 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.507586 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.507684 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.507734 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":40}
I20260806 12:28:54.507921 9182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.504415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41967 (local address 127.6.152.129:47631)
0806 12:28:54.504722 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.504735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.504817 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.505038 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.505040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.505045 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:54.505154 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.505158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.506050 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.506052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.506628 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.506635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.507203 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.507206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.507272 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.507311 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.507615 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.507625 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.507675 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.507741 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":2,"thread_start_us":140,"threads_started":1}
I20260806 12:28:54.514289 8904 raft_consensus.cc:1261] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Refusing update from remote peer bf483819a21540c1864bcef2b9769328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:54.514308 9068 raft_consensus.cc:1261] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Refusing update from remote peer bf483819a21540c1864bcef2b9769328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:54.514665 9153 consensus_queue.cc:1035] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:54.514796 9150 consensus_queue.cc:1035] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:54.554673 8577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:54.554853 8577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:54.557226 8577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Sending DeleteTablet for 3 replicas of tablet 6c68b0f3d11b49809bf6ae10250ef020
I20260806 12:28:54.557811 8882 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c68b0f3d11b49809bf6ae10250ef020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:54 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:37456
I20260806 12:28:54.557909 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c68b0f3d11b49809bf6ae10250ef020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:54 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:54462
I20260806 12:28:54.558086 9044 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c68b0f3d11b49809bf6ae10250ef020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:54 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:44008
I20260806 12:28:54.558456 9189 tablet_replica.cc:330] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: stopping tablet replica
I20260806 12:28:54.558598 9190 tablet_replica.cc:330] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: stopping tablet replica
I20260806 12:28:54.558774 9189 raft_consensus.cc:2229] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:28:54.558804 9190 raft_consensus.cc:2229] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:54.558975 9190 raft_consensus.cc:2258] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:54.559099 9189 raft_consensus.cc:2258] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:54.559307 8577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:28:54.559687 8577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:54.560393 9190 ts_tablet_manager.cc:1905] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:54.560889 9189 ts_tablet_manager.cc:1905] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:54.563861 9189 ts_tablet_manager.cc:1918] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:54.563982 9189 log.cc:1196] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/6c68b0f3d11b49809bf6ae10250ef020
I20260806 12:28:54.564297 9189 ts_tablet_manager.cc:1939] T 6c68b0f3d11b49809bf6ae10250ef020 P bf483819a21540c1864bcef2b9769328: Deleting consensus metadata
I20260806 12:28:54.564916 9190 ts_tablet_manager.cc:1918] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:54.564999 9190 log.cc:1196] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/6c68b0f3d11b49809bf6ae10250ef020
I20260806 12:28:54.565078 8882 tablet_service.cc:1467] Processing CreateTablet for tablet 0b36c14642364c4a96b4c00466fa10f4 (DEFAULT_TABLE table=test-table [id=1591f0e95dd14c67a1a84cf5748793ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:54.565105 9044 tablet_service.cc:1467] Processing CreateTablet for tablet 0b36c14642364c4a96b4c00466fa10f4 (DEFAULT_TABLE table=test-table [id=1591f0e95dd14c67a1a84cf5748793ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:54.565327 9190 ts_tablet_manager.cc:1939] T 6c68b0f3d11b49809bf6ae10250ef020 P 1fd2234e44874b80a170e69fc93e1aba: Deleting consensus metadata
I20260806 12:28:54.565356 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b36c14642364c4a96b4c00466fa10f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.565384 8882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b36c14642364c4a96b4c00466fa10f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.566509 9188 tablet_replica.cc:330] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: stopping tablet replica
I20260806 12:28:54.566797 9188 raft_consensus.cc:2229] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:54.567013 9188 raft_consensus.cc:2258] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:54.567636 9143 tablet_bootstrap.cc:492] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Bootstrap starting.
I20260806 12:28:54.567795 9142 tablet_bootstrap.cc:492] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Bootstrap starting.
I20260806 12:28:54.568567 9143 tablet_bootstrap.cc:654] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.568696 9142 tablet_bootstrap.cc:654] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.569267 8562 catalog_manager.cc:4841] TS bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967): tablet 6c68b0f3d11b49809bf6ae10250ef020 (table test-table [id=9cb1638f571e4468893e5187d7c72eda]) successfully deleted
I20260806 12:28:54.569726 8564 catalog_manager.cc:4841] TS 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505): tablet 6c68b0f3d11b49809bf6ae10250ef020 (table test-table [id=9cb1638f571e4468893e5187d7c72eda]) successfully deleted
I20260806 12:28:54.570003 9143 tablet_bootstrap.cc:492] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: No bootstrap required, opened a new log
I20260806 12:28:54.570076 9143 ts_tablet_manager.cc:1397] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 12:28:54.570313 9188 ts_tablet_manager.cc:1905] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:54.570503 8724 tablet_service.cc:1467] Processing CreateTablet for tablet 0b36c14642364c4a96b4c00466fa10f4 (DEFAULT_TABLE table=test-table [id=1591f0e95dd14c67a1a84cf5748793ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:54.570776 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b36c14642364c4a96b4c00466fa10f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:54.570811 9143 raft_consensus.cc:348] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.570916 9143 raft_consensus.cc:374] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.570947 9143 raft_consensus.cc:729] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.571066 9143 consensus_queue.cc:260] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.571954 9143 ts_tablet_manager.cc:1428] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:28:54.572871 9188 ts_tablet_manager.cc:1918] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:54.572978 9188 log.cc:1196] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/6c68b0f3d11b49809bf6ae10250ef020
I20260806 12:28:54.573323 9188 ts_tablet_manager.cc:1939] T 6c68b0f3d11b49809bf6ae10250ef020 P faacf54df216422eb2e530bc31140285: Deleting consensus metadata
I20260806 12:28:54.574550 9142 tablet_bootstrap.cc:492] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: No bootstrap required, opened a new log
I20260806 12:28:54.574637 9142 ts_tablet_manager.cc:1397] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 12:28:54.575080 9142 raft_consensus.cc:348] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.575245 9142 raft_consensus.cc:374] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.575280 9142 raft_consensus.cc:729] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faacf54df216422eb2e530bc31140285, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.575403 9142 consensus_queue.cc:260] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.575513 9144 tablet_bootstrap.cc:492] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Bootstrap starting.
I20260806 12:28:54.575724 9142 ts_tablet_manager.cc:1428] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:28:54.576248 8564 catalog_manager.cc:4841] TS faacf54df216422eb2e530bc31140285 (127.6.152.130:44331): tablet 6c68b0f3d11b49809bf6ae10250ef020 (table test-table [id=9cb1638f571e4468893e5187d7c72eda]) successfully deleted
I20260806 12:28:54.576459 9144 tablet_bootstrap.cc:654] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:54.577988 9144 tablet_bootstrap.cc:492] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: No bootstrap required, opened a new log
I20260806 12:28:54.578102 9144 ts_tablet_manager.cc:1397] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:28:54.578860 9144 raft_consensus.cc:348] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.578975 9144 raft_consensus.cc:374] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:54.579008 9144 raft_consensus.cc:729] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf483819a21540c1864bcef2b9769328, State: Initialized, Role: FOLLOWER
I20260806 12:28:54.579360 9144 consensus_queue.cc:260] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.579860 9144 ts_tablet_manager.cc:1428] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:28:54.626075 9148 raft_consensus.cc:482] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:54.626246 9148 raft_consensus.cc:504] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.627381 9148 leader_election.cc:288] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967), faacf54df216422eb2e530bc31140285 (127.6.152.130:44331)
I20260806 12:28:54.631971 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.627849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58131 (local address 127.6.152.129:41967)
0806 12:28:54.628019 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.628023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.628038 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:28:54.628096 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.628101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.628164 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.628248 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.628643 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.629187 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.630738 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.631377 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.631502 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.631557 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.631772 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.631841 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 12:28:54.631973 8998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.627588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41967 (local address 127.6.152.131:58131)
0806 12:28:54.627914 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.627929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.628013 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.628316 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.628319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.628335 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.628478 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.628484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.629308 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.629313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.630593 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.630604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.631519 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.631523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.631641 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.631667 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.631747 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.631826 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257}
I20260806 12:28:54.632128 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.627780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:34663 (local address 127.6.152.130:44331)
0806 12:28:54.627913 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.627920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.628176 (+ 256us) server_negotiation.cc:407] Connection header received
0806 12:28:54.628296 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.628299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.628345 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.628428 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:28:54.628745 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.629490 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.630884 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.631421 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.631558 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.631615 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.631921 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.632009 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:28:54.632490 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.627648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:44331 (local address 127.6.152.131:34663)
0806 12:28:54.628066 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.628080 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.628188 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.628457 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.628460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.628473 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:28:54.628633 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.628640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.629609 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.629612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.630763 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.630770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.631683 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.631686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.631788 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.631806 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.632273 (+ 467us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.632362 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":85,"threads_started":1}
I20260806 12:28:54.632699 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b36c14642364c4a96b4c00466fa10f4" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf483819a21540c1864bcef2b9769328" is_pre_election: true
I20260806 12:28:54.632928 8746 raft_consensus.cc:2454] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 0.
I20260806 12:28:54.633076 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b36c14642364c4a96b4c00466fa10f4" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285" is_pre_election: true
I20260806 12:28:54.633240 8904 raft_consensus.cc:2454] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 0.
I20260806 12:28:54.633340 8989 leader_election.cc:302] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd2234e44874b80a170e69fc93e1aba, bf483819a21540c1864bcef2b9769328; no voters:
I20260806 12:28:54.633744 9148 raft_consensus.cc:2781] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:54.633807 9148 raft_consensus.cc:482] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:54.633846 9148 raft_consensus.cc:3037] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.635195 9148 raft_consensus.cc:504] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.635571 9148 leader_election.cc:288] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 election: Requested vote from peers bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967), faacf54df216422eb2e530bc31140285 (127.6.152.130:44331)
I20260806 12:28:54.635994 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b36c14642364c4a96b4c00466fa10f4" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf483819a21540c1864bcef2b9769328"
I20260806 12:28:54.636142 8746 raft_consensus.cc:3037] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.636132 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b36c14642364c4a96b4c00466fa10f4" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285"
I20260806 12:28:54.636240 8904 raft_consensus.cc:3037] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:54.637161 8746 raft_consensus.cc:2454] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 1.
I20260806 12:28:54.637287 8904 raft_consensus.cc:2454] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 1.
I20260806 12:28:54.637598 8991 leader_election.cc:302] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd2234e44874b80a170e69fc93e1aba, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:54.637913 9148 raft_consensus.cc:2781] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:54.638088 9148 raft_consensus.cc:686] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 1 LEADER]: Becoming Leader. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Running, Role: LEADER
I20260806 12:28:54.638322 9148 consensus_queue.cc:237] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:54.640543 8578 catalog_manager.cc:5495] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131). New cstate: current_term: 1 leader_uuid: "1fd2234e44874b80a170e69fc93e1aba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } health_report { overall_health: HEALTHY } } }
I20260806 12:28:54.662845 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.659210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.6.152.131:35505)
0806 12:28:54.659320 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.659327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.659361 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:28:54.659541 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.659543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.659593 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.659680 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:54.659913 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.660377 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.661486 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.661690 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.661760 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.662026 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.662477 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.662574 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.662644 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.662688 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:28:54.672544 9124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.669389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41967 (local address 127.6.152.131:49077)
0806 12:28:54.669545 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.669564 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.669648 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.669893 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.669896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.669903 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:54.670009 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.670014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.670705 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.670710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.671355 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.671363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.671809 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.671813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.671905 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.671945 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.672251 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.672259 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.672334 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.672410 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 12:28:54.672536 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.669501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:49077 (local address 127.6.152.129:41967)
0806 12:28:54.669624 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.669629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.669642 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:28:54.669738 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.669741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.669787 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.669883 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:54.670113 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.670593 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.671469 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.671701 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.671809 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.672082 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.672157 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.672245 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.672357 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.672408 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:28:54.678427 8746 raft_consensus.cc:1261] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Refusing update from remote peer 1fd2234e44874b80a170e69fc93e1aba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:54.678457 8904 raft_consensus.cc:1261] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Refusing update from remote peer 1fd2234e44874b80a170e69fc93e1aba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:54.678956 9148 consensus_queue.cc:1035] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [LEADER]: Connected to new peer: Peer: permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:54.679173 9195 consensus_queue.cc:1035] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:54.697832 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.694412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:39889 (local address 127.6.152.131:35505)
0806 12:28:54.694506 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:54.694510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:54.694575 (+ 65us) server_negotiation.cc:407] Connection header received
0806 12:28:54.694714 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:54.694717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:54.694762 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:54.694837 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:54.695141 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.695866 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.696683 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.697009 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.697109 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.697320 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.697446 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.697518 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:54.697647 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 12:28:54.697696 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:28:54.697836 9182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:54.694350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35505 (local address 127.6.152.129:39889)
0806 12:28:54.694517 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:54.694533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:28:54.694618 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:28:54.694864 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:28:54.694869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:28:54.694881 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:28:54.695000 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.695006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.695968 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.695970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.696575 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:28:54.696584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:28:54.697109 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:28:54.697112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:28:54.697168 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:54.697193 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:28:54.697545 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:28:54.697552 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 12:28:54.697627 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 12:28:54.697710 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
W20260806 12:28:54.698665 9173 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 12:28:56.260509 8578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:56.260668 8578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:56.262933 8578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Sending DeleteTablet for 3 replicas of tablet 0b36c14642364c4a96b4c00466fa10f4
I20260806 12:28:56.263448 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b36c14642364c4a96b4c00466fa10f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:56 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:54462
I20260806 12:28:56.263586 8882 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b36c14642364c4a96b4c00466fa10f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:56 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:37456
I20260806 12:28:56.263664 9044 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b36c14642364c4a96b4c00466fa10f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:56 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:44008
I20260806 12:28:56.264026 9226 tablet_replica.cc:330] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: stopping tablet replica
I20260806 12:28:56.264083 9228 tablet_replica.cc:330] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: stopping tablet replica
I20260806 12:28:56.264214 9226 raft_consensus.cc:2229] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:56.264246 9228 raft_consensus.cc:2229] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:28:56.264379 9226 raft_consensus.cc:2258] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:56.264489 9227 tablet_replica.cc:330] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: stopping tablet replica
I20260806 12:28:56.264582 9228 raft_consensus.cc:2258] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:56.265012 9227 raft_consensus.cc:2229] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:56.265106 9226 ts_tablet_manager.cc:1905] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:56.265200 9227 raft_consensus.cc:2258] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:56.265223 9228 ts_tablet_manager.cc:1905] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:56.265980 9227 ts_tablet_manager.cc:1905] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:56.268056 9228 ts_tablet_manager.cc:1918] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:28:56.268193 9228 log.cc:1196] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/0b36c14642364c4a96b4c00466fa10f4
I20260806 12:28:56.268467 9226 ts_tablet_manager.cc:1918] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:28:56.268573 9228 ts_tablet_manager.cc:1939] T 0b36c14642364c4a96b4c00466fa10f4 P 1fd2234e44874b80a170e69fc93e1aba: Deleting consensus metadata
I20260806 12:28:56.268627 9226 log.cc:1196] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/0b36c14642364c4a96b4c00466fa10f4
I20260806 12:28:56.268936 9226 ts_tablet_manager.cc:1939] T 0b36c14642364c4a96b4c00466fa10f4 P bf483819a21540c1864bcef2b9769328: Deleting consensus metadata
I20260806 12:28:56.269160 9227 ts_tablet_manager.cc:1918] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:28:56.269254 9227 log.cc:1196] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/0b36c14642364c4a96b4c00466fa10f4
I20260806 12:28:56.269344 8564 catalog_manager.cc:4841] TS 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505): tablet 0b36c14642364c4a96b4c00466fa10f4 (table test-table [id=1591f0e95dd14c67a1a84cf5748793ee]) successfully deleted
I20260806 12:28:56.269565 9227 ts_tablet_manager.cc:1939] T 0b36c14642364c4a96b4c00466fa10f4 P faacf54df216422eb2e530bc31140285: Deleting consensus metadata
I20260806 12:28:56.269937 8562 catalog_manager.cc:4841] TS bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967): tablet 0b36c14642364c4a96b4c00466fa10f4 (table test-table [id=1591f0e95dd14c67a1a84cf5748793ee]) successfully deleted
I20260806 12:28:56.270186 8564 catalog_manager.cc:4841] TS faacf54df216422eb2e530bc31140285 (127.6.152.130:44331): tablet 0b36c14642364c4a96b4c00466fa10f4 (table test-table [id=1591f0e95dd14c67a1a84cf5748793ee]) successfully deleted
Aug 06 12:28:56 dist-test-slave-1xrh krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019336, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:28:56 dist-test-slave-1xrh krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019336, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:56.294325 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:56.285659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.6.152.190:35345)
0806 12:28:56.285990 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:56.286002 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:56.286018 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:28:56.286069 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:56.286071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:56.286114 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:56.286218 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:56.286987 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:56.287798 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:56.288699 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:56.288942 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:56.289054 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:56.291425 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:56.291428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:56.291430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:56.291454 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:56.292986 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:56.293346 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:56.293349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:56.293350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:56.293431 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:56.293656 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:56.293658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:56.293659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:56.293827 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:56.293944 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:56.294094 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 12:28:56.294193 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":123,"threads_started":1}
I20260806 12:28:56.296903 8578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:28:56.297523 8578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:56.301987 8724 tablet_service.cc:1467] Processing CreateTablet for tablet 2ddf37129ac04c9b8ae3ef137603a426 (DEFAULT_TABLE table=test-table [id=c04d21c97a5e442dbec17bef623ac9d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:56.302048 8882 tablet_service.cc:1467] Processing CreateTablet for tablet 2ddf37129ac04c9b8ae3ef137603a426 (DEFAULT_TABLE table=test-table [id=c04d21c97a5e442dbec17bef623ac9d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:56.302105 9044 tablet_service.cc:1467] Processing CreateTablet for tablet 2ddf37129ac04c9b8ae3ef137603a426 (DEFAULT_TABLE table=test-table [id=c04d21c97a5e442dbec17bef623ac9d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:56.302299 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ddf37129ac04c9b8ae3ef137603a426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:56.302320 8882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ddf37129ac04c9b8ae3ef137603a426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:56.302357 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ddf37129ac04c9b8ae3ef137603a426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:56.304394 9237 tablet_bootstrap.cc:492] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Bootstrap starting.
I20260806 12:28:56.304841 9238 tablet_bootstrap.cc:492] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Bootstrap starting.
I20260806 12:28:56.305365 9237 tablet_bootstrap.cc:654] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:56.305492 9239 tablet_bootstrap.cc:492] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Bootstrap starting.
I20260806 12:28:56.305873 9238 tablet_bootstrap.cc:654] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:56.306403 9239 tablet_bootstrap.cc:654] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:56.307070 9238 tablet_bootstrap.cc:492] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: No bootstrap required, opened a new log
I20260806 12:28:56.307191 9238 ts_tablet_manager.cc:1397] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:28:56.307312 9237 tablet_bootstrap.cc:492] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: No bootstrap required, opened a new log
I20260806 12:28:56.307399 9237 ts_tablet_manager.cc:1397] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:28:56.307660 9239 tablet_bootstrap.cc:492] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: No bootstrap required, opened a new log
I20260806 12:28:56.307734 9239 ts_tablet_manager.cc:1397] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:56.307654 9238 raft_consensus.cc:348] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.307776 9238 raft_consensus.cc:374] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:56.307817 9238 raft_consensus.cc:729] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Initialized, Role: FOLLOWER
I20260806 12:28:56.307935 9237 raft_consensus.cc:348] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.307991 9238 consensus_queue.cc:260] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.308050 9237 raft_consensus.cc:374] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:56.308084 9237 raft_consensus.cc:729] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faacf54df216422eb2e530bc31140285, State: Initialized, Role: FOLLOWER
I20260806 12:28:56.308122 9239 raft_consensus.cc:348] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.308238 9237 consensus_queue.cc:260] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.308295 9239 raft_consensus.cc:374] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:56.308389 9239 raft_consensus.cc:729] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf483819a21540c1864bcef2b9769328, State: Initialized, Role: FOLLOWER
I20260806 12:28:56.308707 9237 ts_tablet_manager.cc:1428] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:28:56.308820 9239 consensus_queue.cc:260] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.308825 9238 ts_tablet_manager.cc:1428] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:56.309644 9239 ts_tablet_manager.cc:1428] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:28:56.502866 9148 raft_consensus.cc:482] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:56.503050 9148 raft_consensus.cc:504] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.503531 9148 leader_election.cc:288] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967), faacf54df216422eb2e530bc31140285 (127.6.152.130:44331)
I20260806 12:28:56.504017 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf37129ac04c9b8ae3ef137603a426" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285" is_pre_election: true
I20260806 12:28:56.504040 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf37129ac04c9b8ae3ef137603a426" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf483819a21540c1864bcef2b9769328" is_pre_election: true
I20260806 12:28:56.504179 8904 raft_consensus.cc:2454] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 0.
I20260806 12:28:56.504199 8746 raft_consensus.cc:2454] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 0.
I20260806 12:28:56.504523 8991 leader_election.cc:302] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd2234e44874b80a170e69fc93e1aba, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:56.504689 9148 raft_consensus.cc:2781] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:56.504750 9148 raft_consensus.cc:482] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:56.504791 9148 raft_consensus.cc:3037] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:56.505811 9148 raft_consensus.cc:504] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.506142 9148 leader_election.cc:288] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 election: Requested vote from peers bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967), faacf54df216422eb2e530bc31140285 (127.6.152.130:44331)
I20260806 12:28:56.506433 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf37129ac04c9b8ae3ef137603a426" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf483819a21540c1864bcef2b9769328"
I20260806 12:28:56.506480 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ddf37129ac04c9b8ae3ef137603a426" candidate_uuid: "1fd2234e44874b80a170e69fc93e1aba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285"
I20260806 12:28:56.506564 8746 raft_consensus.cc:3037] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:56.506592 8904 raft_consensus.cc:3037] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:56.507525 8904 raft_consensus.cc:2454] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 1.
I20260806 12:28:56.507727 8746 raft_consensus.cc:2454] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd2234e44874b80a170e69fc93e1aba in term 1.
I20260806 12:28:56.507839 8991 leader_election.cc:302] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd2234e44874b80a170e69fc93e1aba, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:56.508031 9148 raft_consensus.cc:2781] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:56.508167 9148 raft_consensus.cc:686] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 1 LEADER]: Becoming Leader. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Running, Role: LEADER
I20260806 12:28:56.508307 9148 consensus_queue.cc:237] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.510044 8576 catalog_manager.cc:5495] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131). New cstate: current_term: 1 leader_uuid: "1fd2234e44874b80a170e69fc93e1aba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } health_report { overall_health: HEALTHY } } }
I20260806 12:28:56.549633 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:56.546652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.6.152.131:35505)
0806 12:28:56.546929 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:56.546936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:56.546948 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:28:56.546986 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:56.546989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:56.547027 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:56.547099 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:56.547363 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:56.547870 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:56.548688 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:56.548898 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:56.548980 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:56.549204 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:56.549271 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:56.549321 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:56.549415 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 12:28:56.549457 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":118,"threads_started":1}
I20260806 12:28:56.555562 8904 raft_consensus.cc:1261] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Refusing update from remote peer 1fd2234e44874b80a170e69fc93e1aba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:56.555588 8746 raft_consensus.cc:1261] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Refusing update from remote peer 1fd2234e44874b80a170e69fc93e1aba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:56.556072 9219 consensus_queue.cc:1035] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [LEADER]: Connected to new peer: Peer: permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:56.556209 9148 consensus_queue.cc:1035] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:56.586234 8577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:56.586383 8577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:56.588291 8577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Sending DeleteTablet for 3 replicas of tablet 2ddf37129ac04c9b8ae3ef137603a426
I20260806 12:28:56.588797 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ddf37129ac04c9b8ae3ef137603a426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:56 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:54462
I20260806 12:28:56.588833 8882 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ddf37129ac04c9b8ae3ef137603a426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:56 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:37456
I20260806 12:28:56.588989 9226 tablet_replica.cc:330] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: stopping tablet replica
I20260806 12:28:56.589109 9226 raft_consensus.cc:2229] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:56.589231 9226 raft_consensus.cc:2258] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:56.589416 9227 tablet_replica.cc:330] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: stopping tablet replica
I20260806 12:28:56.589552 9227 raft_consensus.cc:2229] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:56.589660 9227 raft_consensus.cc:2258] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:56.589572 8577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 12:28:56.589820 9044 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ddf37129ac04c9b8ae3ef137603a426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:56 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:44008
I20260806 12:28:56.589946 9226 ts_tablet_manager.cc:1905] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:56.589986 9228 tablet_replica.cc:330] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: stopping tablet replica
W20260806 12:28:56.589977 8577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:28:56.590090 9228 raft_consensus.cc:2229] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:28:56.590281 9228 raft_consensus.cc:2258] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:56.590691 9227 ts_tablet_manager.cc:1905] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:56.590930 9228 ts_tablet_manager.cc:1905] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:56.592767 9226 ts_tablet_manager.cc:1918] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:56.592833 9226 log.cc:1196] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/2ddf37129ac04c9b8ae3ef137603a426
I20260806 12:28:56.593086 9226 ts_tablet_manager.cc:1939] T 2ddf37129ac04c9b8ae3ef137603a426 P bf483819a21540c1864bcef2b9769328: Deleting consensus metadata
I20260806 12:28:56.593407 9227 ts_tablet_manager.cc:1918] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:56.593478 9227 log.cc:1196] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/2ddf37129ac04c9b8ae3ef137603a426
I20260806 12:28:56.593607 9228 ts_tablet_manager.cc:1918] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:28:56.593730 8562 catalog_manager.cc:4841] TS bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967): tablet 2ddf37129ac04c9b8ae3ef137603a426 (table test-table [id=c04d21c97a5e442dbec17bef623ac9d4]) successfully deleted
I20260806 12:28:56.593776 9227 ts_tablet_manager.cc:1939] T 2ddf37129ac04c9b8ae3ef137603a426 P faacf54df216422eb2e530bc31140285: Deleting consensus metadata
I20260806 12:28:56.593989 9228 log.cc:1196] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/2ddf37129ac04c9b8ae3ef137603a426
I20260806 12:28:56.594511 8564 catalog_manager.cc:4841] TS faacf54df216422eb2e530bc31140285 (127.6.152.130:44331): tablet 2ddf37129ac04c9b8ae3ef137603a426 (table test-table [id=c04d21c97a5e442dbec17bef623ac9d4]) successfully deleted
I20260806 12:28:56.594743 8724 tablet_service.cc:1467] Processing CreateTablet for tablet fcdb399b90b349838182a4b534866468 (DEFAULT_TABLE table=test-table [id=2229a2a4134c437eadbd494db9b738c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:56.595010 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdb399b90b349838182a4b534866468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:56.595005 8882 tablet_service.cc:1467] Processing CreateTablet for tablet fcdb399b90b349838182a4b534866468 (DEFAULT_TABLE table=test-table [id=2229a2a4134c437eadbd494db9b738c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:56.595291 8882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdb399b90b349838182a4b534866468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:56.596957 9239 tablet_bootstrap.cc:492] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328: Bootstrap starting.
I20260806 12:28:56.596989 9237 tablet_bootstrap.cc:492] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Bootstrap starting.
I20260806 12:28:56.595005 9044 tablet_service.cc:1467] Processing CreateTablet for tablet fcdb399b90b349838182a4b534866468 (DEFAULT_TABLE table=test-table [id=2229a2a4134c437eadbd494db9b738c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:28:56.597460 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcdb399b90b349838182a4b534866468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:56.597775 9239 tablet_bootstrap.cc:654] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:56.597949 9237 tablet_bootstrap.cc:654] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:56.598985 9239 tablet_bootstrap.cc:492] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328: No bootstrap required, opened a new log
I20260806 12:28:56.599104 9239 ts_tablet_manager.cc:1397] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:28:56.594260 9228 ts_tablet_manager.cc:1939] T 2ddf37129ac04c9b8ae3ef137603a426 P 1fd2234e44874b80a170e69fc93e1aba: Deleting consensus metadata
I20260806 12:28:56.599565 9239 raft_consensus.cc:348] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.599684 9239 raft_consensus.cc:374] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:56.599700 9238 tablet_bootstrap.cc:492] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Bootstrap starting.
I20260806 12:28:56.599722 9239 raft_consensus.cc:729] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf483819a21540c1864bcef2b9769328, State: Initialized, Role: FOLLOWER
I20260806 12:28:56.599844 9239 consensus_queue.cc:260] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.600155 9239 ts_tablet_manager.cc:1428] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:28:56.600449 9238 tablet_bootstrap.cc:654] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:56.601138 8564 catalog_manager.cc:4841] TS 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505): tablet 2ddf37129ac04c9b8ae3ef137603a426 (table test-table [id=c04d21c97a5e442dbec17bef623ac9d4]) successfully deleted
I20260806 12:28:56.601608 9238 tablet_bootstrap.cc:492] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: No bootstrap required, opened a new log
I20260806 12:28:56.601642 9237 tablet_bootstrap.cc:492] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: No bootstrap required, opened a new log
I20260806 12:28:56.601719 9237 ts_tablet_manager.cc:1397] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 12:28:56.601720 9238 ts_tablet_manager.cc:1397] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:56.602141 9238 raft_consensus.cc:348] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.602264 9238 raft_consensus.cc:374] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:56.602308 9238 raft_consensus.cc:729] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd2234e44874b80a170e69fc93e1aba, State: Initialized, Role: FOLLOWER
I20260806 12:28:56.602438 9238 consensus_queue.cc:260] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.602738 9238 ts_tablet_manager.cc:1428] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:28:56.603631 9237 raft_consensus.cc:348] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.603750 9237 raft_consensus.cc:374] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:56.603807 9237 raft_consensus.cc:729] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faacf54df216422eb2e530bc31140285, State: Initialized, Role: FOLLOWER
I20260806 12:28:56.603952 9237 consensus_queue.cc:260] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.604256 9237 ts_tablet_manager.cc:1428] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:28:56.719323 9206 raft_consensus.cc:482] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:28:56.719506 9206 raft_consensus.cc:504] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.719974 9206 leader_election.cc:288] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faacf54df216422eb2e530bc31140285 (127.6.152.130:44331), 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:56.720394 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdb399b90b349838182a4b534866468" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285" is_pre_election: true
I20260806 12:28:56.720431 9066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdb399b90b349838182a4b534866468" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd2234e44874b80a170e69fc93e1aba" is_pre_election: true
I20260806 12:28:56.720572 8904 raft_consensus.cc:2454] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 0.
I20260806 12:28:56.720578 9066 raft_consensus.cc:2454] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 0.
I20260806 12:28:56.720949 8675 leader_election.cc:302] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd2234e44874b80a170e69fc93e1aba, bf483819a21540c1864bcef2b9769328; no voters:
I20260806 12:28:56.721141 9206 raft_consensus.cc:2781] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:28:56.721202 9206 raft_consensus.cc:482] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:28:56.721254 9206 raft_consensus.cc:3037] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:56.722313 9206 raft_consensus.cc:504] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.722653 9206 leader_election.cc:288] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 election: Requested vote from peers faacf54df216422eb2e530bc31140285 (127.6.152.130:44331), 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505)
I20260806 12:28:56.723081 8904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdb399b90b349838182a4b534866468" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacf54df216422eb2e530bc31140285"
I20260806 12:28:56.723109 9066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcdb399b90b349838182a4b534866468" candidate_uuid: "bf483819a21540c1864bcef2b9769328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd2234e44874b80a170e69fc93e1aba"
I20260806 12:28:56.723235 8904 raft_consensus.cc:3037] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:56.723235 9066 raft_consensus.cc:3037] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:56.724422 8904 raft_consensus.cc:2454] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 1.
I20260806 12:28:56.724422 9066 raft_consensus.cc:2454] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf483819a21540c1864bcef2b9769328 in term 1.
I20260806 12:28:56.724732 8675 leader_election.cc:302] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf483819a21540c1864bcef2b9769328, faacf54df216422eb2e530bc31140285; no voters:
I20260806 12:28:56.724895 9206 raft_consensus.cc:2781] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:56.724987 9206 raft_consensus.cc:686] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 1 LEADER]: Becoming Leader. State: Replica: bf483819a21540c1864bcef2b9769328, State: Running, Role: LEADER
I20260806 12:28:56.725143 9206 consensus_queue.cc:237] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } }
I20260806 12:28:56.727001 8577 catalog_manager.cc:5495] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf483819a21540c1864bcef2b9769328 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "bf483819a21540c1864bcef2b9769328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf483819a21540c1864bcef2b9769328" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 41967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 } health_report { overall_health: UNKNOWN } } }
I20260806 12:28:56.761075 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:56.757868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54488 (local address 127.6.152.129:41967)
0806 12:28:56.758184 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:56.758194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:56.758206 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:28:56.758251 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:56.758253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:56.758290 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:56.758375 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:28:56.758664 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:56.759229 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:56.760029 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:56.760253 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:56.760365 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:56.760611 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:28:56.760690 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:28:56.760767 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:56.760882 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 12:28:56.760932 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":134,"threads_started":1}
I20260806 12:28:56.766671 9066 raft_consensus.cc:1261] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Refusing update from remote peer bf483819a21540c1864bcef2b9769328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:56.766709 8904 raft_consensus.cc:1261] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Refusing update from remote peer bf483819a21540c1864bcef2b9769328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:28:56.767112 9206 consensus_queue.cc:1035] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faacf54df216422eb2e530bc31140285" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:28:56.767395 9211 consensus_queue.cc:1035] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd2234e44874b80a170e69fc93e1aba" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 12:28:56.779853 9204 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 12:28:58.329648 8576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:58.329797 8576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:28:58.332064 8576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0a5c026859f4c089b1e327508cf79c0: Sending DeleteTablet for 3 replicas of tablet fcdb399b90b349838182a4b534866468
I20260806 12:28:58.332580 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdb399b90b349838182a4b534866468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:58 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:54462
I20260806 12:28:58.332617 8882 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdb399b90b349838182a4b534866468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:58 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:37456
I20260806 12:28:58.332743 9044 tablet_service.cc:1514] Processing DeleteTablet for tablet fcdb399b90b349838182a4b534866468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:28:58 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:44008
I20260806 12:28:58.332960 9280 tablet_replica.cc:330] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328: stopping tablet replica
I20260806 12:28:58.333077 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8655
I20260806 12:28:58.333097 9280 raft_consensus.cc:2229] T fcdb399b90b349838182a4b534866468 P bf483819a21540c1864bcef2b9769328 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:28:58.333114 9281 tablet_replica.cc:330] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: stopping tablet replica
I20260806 12:28:58.335402 9279 tablet_replica.cc:330] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: stopping tablet replica
I20260806 12:28:58.335562 9279 raft_consensus.cc:2229] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:58.335682 9279 raft_consensus.cc:2258] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:58.336122 9281 raft_consensus.cc:2229] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:28:58.336248 9281 raft_consensus.cc:2258] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:28:58.336347 9279 ts_tablet_manager.cc:1905] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:58.336900 9281 ts_tablet_manager.cc:1905] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:28:58.339077 9279 ts_tablet_manager.cc:1918] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:28:58.339197 9279 log.cc:1196] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/fcdb399b90b349838182a4b534866468
I20260806 12:28:58.339501 9279 ts_tablet_manager.cc:1939] T fcdb399b90b349838182a4b534866468 P faacf54df216422eb2e530bc31140285: Deleting consensus metadata
I20260806 12:28:58.339648 9281 ts_tablet_manager.cc:1918] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 12:28:58.339727 9281 log.cc:1196] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/fcdb399b90b349838182a4b534866468
I20260806 12:28:58.339989 9281 ts_tablet_manager.cc:1939] T fcdb399b90b349838182a4b534866468 P 1fd2234e44874b80a170e69fc93e1aba: Deleting consensus metadata
I20260806 12:28:58.340260 8564 catalog_manager.cc:4841] TS faacf54df216422eb2e530bc31140285 (127.6.152.130:44331): tablet fcdb399b90b349838182a4b534866468 (table test-table [id=2229a2a4134c437eadbd494db9b738c6]) successfully deleted
I20260806 12:28:58.340668 8564 catalog_manager.cc:4841] TS 1fd2234e44874b80a170e69fc93e1aba (127.6.152.131:35505): tablet fcdb399b90b349838182a4b534866468 (table test-table [id=2229a2a4134c437eadbd494db9b738c6]) successfully deleted
W20260806 12:28:58.343365 8562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:41967 (error 108)
I20260806 12:28:58.344192 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8809
I20260806 12:28:58.345585 9283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:58.344078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:41967 (local address 127.0.0.1:54500)
0806 12:28:58.345333 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0806 12:28:58.345444 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
W20260806 12:28:58.345808 8562 catalog_manager.cc:4584] TS bf483819a21540c1864bcef2b9769328 (127.6.152.129:41967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcdb399b90b349838182a4b534866468: Network error: Client connection negotiation failed: client connection to 127.6.152.129:41967: connect: Connection refused (error 111)
I20260806 12:28:58.351775 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8969
I20260806 12:28:58.359073 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 8545
2026-08-06T12:28:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6324 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:28:58 dist-test-slave-1xrh krb5kdc[9286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:28:58 dist-test-slave-1xrh krb5kdc[9286](info): set up 2 sockets
Aug 06 12:28:58 dist-test-slave-1xrh krb5kdc[9286](info): commencing operation
krb5kdc: starting...
W20260806 12:28:59.435601 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:28:59 dist-test-slave-1xrh krb5kdc[9286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:28:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:28:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:28:59.557569 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33537
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:33537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:59.666446 9300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:59.666693 9300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:59.666731 9300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:59.670933 9300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:28:59.671003 9300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:59.671025 9300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:59.671041 9300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:28:59.671056 9300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:28:59.676079 9300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:33537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:59.676937 9300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:59.677845 9300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:59.686182 9307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:59.686225 9306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:59.686363 9300 server_base.cc:1034] running on GCE node
W20260806 12:28:59.686299 9309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:59.687054 9300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:59.688141 9300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:59.689332 9300 hybrid_clock.cc:648] HybridClock initialized: now 1786019339689302 us; error 41 us; skew 500 ppm
Aug 06 12:28:59 dist-test-slave-1xrh krb5kdc[9286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:59.692324 9300 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:28:59.693099 9300 webserver.cc:460] Webserver started at http://127.6.152.190:36845/ using document root <none> and password file <none>
I20260806 12:28:59.693595 9300 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:59.693652 9300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:59.693801 9300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:59.695742 9300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "8fe78c25ae09493682204e0bd08a7f90"
format_stamp: "Formatted at 2026-08-06 12:28:59 on dist-test-slave-1xrh"
server_key: "23998add2bb2115dba6467cad1bcff7b"
server_key_iv: "4dd67faae85cce385d14aead6ee1aa9e"
server_key_version: "encryptionkey@0"
I20260806 12:28:59.696267 9300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "8fe78c25ae09493682204e0bd08a7f90"
format_stamp: "Formatted at 2026-08-06 12:28:59 on dist-test-slave-1xrh"
server_key: "23998add2bb2115dba6467cad1bcff7b"
server_key_iv: "4dd67faae85cce385d14aead6ee1aa9e"
server_key_version: "encryptionkey@0"
I20260806 12:28:59.699965 9300 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:28:59.702203 9316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:59.703423 9300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:28:59.703626 9300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "8fe78c25ae09493682204e0bd08a7f90"
format_stamp: "Formatted at 2026-08-06 12:28:59 on dist-test-slave-1xrh"
server_key: "23998add2bb2115dba6467cad1bcff7b"
server_key_iv: "4dd67faae85cce385d14aead6ee1aa9e"
server_key_version: "encryptionkey@0"
I20260806 12:28:59.703763 9300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:28:59.728091 9300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:28:59.730968 9300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:28:59.731190 9300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:28:59.738268 9300 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:33537
I20260806 12:28:59.738552 9371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:33537 every 8 connection(s)
I20260806 12:28:59.739267 9300 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:28:59.742013 9372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:28:59.742933 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 9300
I20260806 12:28:59.743067 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:28:59.743425 6754 external_mini_cluster.cc:1428] Setting key 09b3a0f701983b77904e4de0fb96d551
I20260806 12:28:59.748240 9372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90: Bootstrap starting.
I20260806 12:28:59.750408 9372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90: Neither blocks nor log segments found. Creating new log.
I20260806 12:28:59.751204 9372 log.cc:824] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:28:59 dist-test-slave-1xrh krb5kdc[9286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:28:59.753449 9372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90: No bootstrap required, opened a new log
I20260806 12:28:59.758306 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:28:59.744916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.6.152.190:33537)
0806 12:28:59.745350 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 12:28:59.745366 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:28:59.745402 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:28:59.746047 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:28:59.746068 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:28:59.746386 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:28:59.746725 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:28:59.747595 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:59.748613 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:59.749479 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:28:59.749954 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:28:59.750116 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:28:59.753105 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:28:59.753110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:28:59.753123 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:28:59.753148 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:28:59.755106 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:59.755583 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:59.755591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:59.755597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:59.755756 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:28:59.756037 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:28:59.756041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:28:59.756043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:28:59.756575 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:28:59.756766 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 12:28:59.757024 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 12:28:59.757330 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":125,"threads_started":1}
I20260806 12:28:59.758293 9372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe78c25ae09493682204e0bd08a7f90" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33537 } }
I20260806 12:28:59.758584 9372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:28:59.758646 9372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe78c25ae09493682204e0bd08a7f90, State: Initialized, Role: FOLLOWER
I20260806 12:28:59.759111 9372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe78c25ae09493682204e0bd08a7f90" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33537 } }
I20260806 12:28:59.759301 9372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:28:59.759372 9372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:28:59.759467 9372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:28:59.760406 9372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe78c25ae09493682204e0bd08a7f90" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33537 } }
I20260806 12:28:59.760753 9372 leader_election.cc:302] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fe78c25ae09493682204e0bd08a7f90; no voters:
I20260806 12:28:59.761065 9372 leader_election.cc:288] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:28:59.761142 9377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:28:59.761407 9377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe78c25ae09493682204e0bd08a7f90, State: Running, Role: LEADER
I20260806 12:28:59.761718 9377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe78c25ae09493682204e0bd08a7f90" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33537 } }
I20260806 12:28:59.762292 9372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:28:59.763244 9378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fe78c25ae09493682204e0bd08a7f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe78c25ae09493682204e0bd08a7f90" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33537 } } }
I20260806 12:28:59.763396 9378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:59.764149 9386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:28:59.764268 9379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe78c25ae09493682204e0bd08a7f90. Latest consensus state: current_term: 1 leader_uuid: "8fe78c25ae09493682204e0bd08a7f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe78c25ae09493682204e0bd08a7f90" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33537 } } }
I20260806 12:28:59.764437 9379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90 [sys.catalog]: This master's current role is: LEADER
I20260806 12:28:59.766788 9386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:28:59.772790 9386 catalog_manager.cc:1347] Generated new cluster ID: f6260cdff0484dfe80b6dde150a83704
I20260806 12:28:59.772867 9386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:28:59.785648 9386 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:28:59.786581 9386 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:28:59.792330 9386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fe78c25ae09493682204e0bd08a7f90: Generated new TSK 0
I20260806 12:28:59.792980 9386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:28:59.831175 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33537
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:28:59.945605 9400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:28:59.945879 9400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:28:59.945955 9400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:28:59.949945 9400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:28:59.950032 9400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:28:59.950119 9400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:28:59.955261 9400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:28:59.956259 9400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:28:59.957252 9400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:28:59.966806 9408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:59.966787 9405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:28:59.966791 9406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:28:59.967144 9400 server_base.cc:1034] running on GCE node
I20260806 12:28:59.967525 9400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:28:59.968153 9400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:28:59.969337 9400 hybrid_clock.cc:648] HybridClock initialized: now 1786019339969324 us; error 25 us; skew 500 ppm
Aug 06 12:28:59 dist-test-slave-1xrh krb5kdc[9286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:28:59.972239 9400 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:28:59.973135 9400 webserver.cc:460] Webserver started at http://127.6.152.129:42357/ using document root <none> and password file <none>
I20260806 12:28:59.973666 9400 fs_manager.cc:359] Metadata directory not provided
I20260806 12:28:59.973758 9400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:28:59.973963 9400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:28:59.976042 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "0a644b7e92e349be946f17ead5f13a5b"
format_stamp: "Formatted at 2026-08-06 12:28:59 on dist-test-slave-1xrh"
server_key: "ff1c5597c65f4126c0cde2e627556a95"
server_key_iv: "ddc5072a0e040b8f85337e93ca72dd42"
server_key_version: "encryptionkey@0"
I20260806 12:28:59.976632 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "0a644b7e92e349be946f17ead5f13a5b"
format_stamp: "Formatted at 2026-08-06 12:28:59 on dist-test-slave-1xrh"
server_key: "ff1c5597c65f4126c0cde2e627556a95"
server_key_iv: "ddc5072a0e040b8f85337e93ca72dd42"
server_key_version: "encryptionkey@0"
I20260806 12:28:59.980239 9400 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 12:28:59.982712 9415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:28:59.983963 9400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:28:59.984098 9400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "0a644b7e92e349be946f17ead5f13a5b"
format_stamp: "Formatted at 2026-08-06 12:28:59 on dist-test-slave-1xrh"
server_key: "ff1c5597c65f4126c0cde2e627556a95"
server_key_iv: "ddc5072a0e040b8f85337e93ca72dd42"
server_key_version: "encryptionkey@0"
I20260806 12:28:59.984203 9400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:00.012377 9400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:00.016206 9400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:00.016454 9400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:00.017136 9400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:00.018157 9400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:00.018236 9400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.018327 9400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:00.018384 9400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.028323 9400 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:38171
I20260806 12:29:00.028344 9536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:38171 every 8 connection(s)
I20260806 12:29:00.029279 9400 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:00.038172 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 9400
I20260806 12:29:00.038275 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:00.038555 6754 external_mini_cluster.cc:1428] Setting key d5367fbdec756b0ceae7c8cc0d7f40bf
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:00.043860 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:00.031278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:59957 (local address 127.6.152.190:33537)
0806 12:29:00.031444 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:00.031449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:00.032341 (+ 892us) server_negotiation.cc:407] Connection header received
0806 12:29:00.033635 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:00.033638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:00.033689 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:00.033766 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:00.034951 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.035584 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.036711 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.036962 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.037045 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:00.039635 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:00.039639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:00.039641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:00.039665 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:00.041507 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:00.042032 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:00.042038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:00.042043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:00.042145 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:00.042611 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:00.042614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:00.042615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:00.042791 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:00.042869 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:00.043579 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 12:29:00.043708 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:29:00.045025 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:00.031693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33537 (local address 127.6.152.129:59957)
0806 12:29:00.032163 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:00.032207 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:00.033421 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:00.033952 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:00.033958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:00.034311 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:00.034771 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:00.034783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.035715 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.035718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:00.036588 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:00.036597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.037097 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.037099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:00.037241 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:00.037795 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:00.037811 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:00.039476 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:00.041630 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:00.041636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:00.041649 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:00.041906 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:00.042263 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:00.042268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:00.042271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:00.042503 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:00.042897 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:00.042905 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:00.043261 (+ 356us) client_negotiation.cc:769] Sending connection context
0806 12:29:00.043511 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 12:29:00.043903 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":120,"threads_started":1}
I20260806 12:29:00.046368 9537 heartbeater.cc:347] Connected to a master server at 127.6.152.190:33537
I20260806 12:29:00.046619 9537 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:00.047161 9537 heartbeater.cc:511] Master 127.6.152.190:33537 requested a full tablet report, sending...
I20260806 12:29:00.048627 9333 ts_manager.cc:194] Registered new tserver with Master: 0a644b7e92e349be946f17ead5f13a5b (127.6.152.129:38171)
I20260806 12:29:00.049728 9333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:59957
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:00.070428 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33537
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:00.186825 9542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:00.187106 9542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:00.187208 9542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:00.191401 9542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:00.191463 9542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:00.191558 9542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:00.196910 9542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:00.197889 9542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:00.198892 9542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:00.208142 9551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:00.208122 9547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:00.208249 9542 server_base.cc:1034] running on GCE node
W20260806 12:29:00.208436 9548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:00.208863 9542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:00.209486 9542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:00.210649 9542 hybrid_clock.cc:648] HybridClock initialized: now 1786019340210591 us; error 74 us; skew 500 ppm
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:00.213569 9542 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:00.214418 9542 webserver.cc:460] Webserver started at http://127.6.152.130:42827/ using document root <none> and password file <none>
I20260806 12:29:00.214967 9542 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:00.215068 9542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:00.215323 9542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:00.217365 9542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "e9401390597b49b3840dd6d9c4d990b3"
format_stamp: "Formatted at 2026-08-06 12:29:00 on dist-test-slave-1xrh"
server_key: "f981ea91781a4a7be6766bc229d9d841"
server_key_iv: "bbb1e291be1cd0308dbc19af456bae72"
server_key_version: "encryptionkey@0"
I20260806 12:29:00.217924 9542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "e9401390597b49b3840dd6d9c4d990b3"
format_stamp: "Formatted at 2026-08-06 12:29:00 on dist-test-slave-1xrh"
server_key: "f981ea91781a4a7be6766bc229d9d841"
server_key_iv: "bbb1e291be1cd0308dbc19af456bae72"
server_key_version: "encryptionkey@0"
I20260806 12:29:00.221536 9542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:00.223872 9559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.225018 9542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:00.225160 9542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "e9401390597b49b3840dd6d9c4d990b3"
format_stamp: "Formatted at 2026-08-06 12:29:00 on dist-test-slave-1xrh"
server_key: "f981ea91781a4a7be6766bc229d9d841"
server_key_iv: "bbb1e291be1cd0308dbc19af456bae72"
server_key_version: "encryptionkey@0"
I20260806 12:29:00.225267 9542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:00.251629 9542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:00.254815 9542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:00.255019 9542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:00.255632 9542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:00.256438 9542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:00.256507 9542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.256595 9542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:00.256641 9542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.265789 9542 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:44477
I20260806 12:29:00.265812 9679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:44477 every 8 connection(s)
I20260806 12:29:00.266607 9542 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:00.267201 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 9542
I20260806 12:29:00.267307 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:00.267575 6754 external_mini_cluster.cc:1428] Setting key d3abc0bb52306051cc5c41e803f3f26b
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:29:00.281483 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:00.268456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:33755 (local address 127.6.152.190:33537)
0806 12:29:00.268688 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:00.268696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:00.269664 (+ 968us) server_negotiation.cc:407] Connection header received
0806 12:29:00.271586 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:00.271589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:00.271645 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:00.271750 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:00.272864 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.273398 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.274530 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.274758 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.274868 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:00.277601 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:00.277604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:00.277606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:00.277630 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:00.279173 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:00.279704 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:00.279708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:00.279728 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:00.279807 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:00.280273 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:00.280276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:00.280277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:00.280509 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:00.280641 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:00.281254 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 12:29:00.281363 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 12:29:00.282600 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:00.268927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33537 (local address 127.6.152.130:33755)
0806 12:29:00.269493 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:00.269552 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:00.271359 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:00.271926 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:00.271933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:00.272265 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:00.272700 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:00.272712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.273533 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.273537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:00.274403 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:00.274413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.274915 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.274919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:00.275083 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:00.275713 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:00.275729 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:00.277391 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:00.279295 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:00.279302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:00.279323 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:00.279581 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:00.279952 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:00.279957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:00.279959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:00.280098 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:00.280638 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:00.280645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:00.280961 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 12:29:00.281202 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 12:29:00.281575 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":148,"threads_started":1}
I20260806 12:29:00.283828 9680 heartbeater.cc:347] Connected to a master server at 127.6.152.190:33537
I20260806 12:29:00.284063 9680 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:00.284533 9680 heartbeater.cc:511] Master 127.6.152.190:33537 requested a full tablet report, sending...
I20260806 12:29:00.285610 9333 ts_manager.cc:194] Registered new tserver with Master: e9401390597b49b3840dd6d9c4d990b3 (127.6.152.130:44477)
I20260806 12:29:00.286170 9333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:33755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:00.298302 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33537
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:00.414109 9685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:00.414366 9685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:00.414409 9685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:00.418641 9685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:00.418691 9685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:00.418762 9685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:00.423768 9685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:00.424681 9685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:00.425580 9685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:00.435103 9693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:00.435146 9695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:00.435335 9685 server_base.cc:1034] running on GCE node
W20260806 12:29:00.435101 9692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:00.435761 9685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:00.436328 9685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:00.437481 9685 hybrid_clock.cc:648] HybridClock initialized: now 1786019340437469 us; error 24 us; skew 500 ppm
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:00.440243 9685 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:00.440980 9685 webserver.cc:460] Webserver started at http://127.6.152.131:42035/ using document root <none> and password file <none>
I20260806 12:29:00.441470 9685 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:00.441527 9685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:00.441685 9685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:00.443631 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "a74e939b15f74b16a172194bebe2d8ca"
format_stamp: "Formatted at 2026-08-06 12:29:00 on dist-test-slave-1xrh"
server_key: "ce488bac3c90768e2e3263abe23e1c49"
server_key_iv: "cad415e4f0248cbaf564e991b4f23604"
server_key_version: "encryptionkey@0"
I20260806 12:29:00.444202 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "a74e939b15f74b16a172194bebe2d8ca"
format_stamp: "Formatted at 2026-08-06 12:29:00 on dist-test-slave-1xrh"
server_key: "ce488bac3c90768e2e3263abe23e1c49"
server_key_iv: "cad415e4f0248cbaf564e991b4f23604"
server_key_version: "encryptionkey@0"
I20260806 12:29:00.447854 9685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:00.450151 9702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.451314 9685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:00.451428 9685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "a74e939b15f74b16a172194bebe2d8ca"
format_stamp: "Formatted at 2026-08-06 12:29:00 on dist-test-slave-1xrh"
server_key: "ce488bac3c90768e2e3263abe23e1c49"
server_key_iv: "cad415e4f0248cbaf564e991b4f23604"
server_key_version: "encryptionkey@0"
I20260806 12:29:00.451525 9685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:00.467690 9685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:00.470742 9685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:00.470899 9685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:00.471536 9685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:00.472306 9685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:00.472353 9685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.472420 9685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:00.472463 9685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:00.481173 9685 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:46403
I20260806 12:29:00.481197 9820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:46403 every 8 connection(s)
I20260806 12:29:00.482055 9685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:00.484669 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 9685
I20260806 12:29:00.484798 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:00.485129 6754 external_mini_cluster.cc:1428] Setting key e462a18616ba5ca404184981c8143663
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:00.495235 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:00.483854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:46377 (local address 127.6.152.190:33537)
0806 12:29:00.483983 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:00.483992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:00.484839 (+ 847us) server_negotiation.cc:407] Connection header received
0806 12:29:00.486422 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:00.486425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:00.486483 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:00.486569 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:00.487773 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.488305 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.489379 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.489599 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.489680 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:00.492086 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:00.492089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:00.492091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:00.492115 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:00.493542 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:00.493966 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:00.493968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:00.493969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:00.494040 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:00.494305 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:00.494307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:00.494308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:00.494469 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:00.494542 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:00.494968 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 12:29:00.495059 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:29:00.496011 9824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:00.484223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33537 (local address 127.6.152.131:46377)
0806 12:29:00.484678 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:00.484719 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:00.486224 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:00.486731 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:00.486737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:00.487099 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:00.487652 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:00.487666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.488409 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.488411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:00.489272 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:00.489281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:00.489737 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:00.489739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:00.489874 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:00.490398 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:00.490414 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:00.491938 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:00.493638 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:00.493642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:00.493652 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:00.493879 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:00.494104 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:00.494106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:00.494107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:00.494228 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:00.494562 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:00.494567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:00.494767 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 12:29:00.494914 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 12:29:00.495203 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":145,"threads_started":1}
I20260806 12:29:00.497089 9822 heartbeater.cc:347] Connected to a master server at 127.6.152.190:33537
I20260806 12:29:00.497293 9822 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:00.497797 9822 heartbeater.cc:511] Master 127.6.152.190:33537 requested a full tablet report, sending...
I20260806 12:29:00.498924 9332 ts_manager.cc:194] Registered new tserver with Master: a74e939b15f74b16a172194bebe2d8ca (127.6.152.131:46403)
I20260806 12:29:00.499537 9332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:46377
I20260806 12:29:00.510424 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 12:29:00.523845 9375 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:00.522288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.6.152.190:33537)
0806 12:29:00.522424 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:00.522427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:00.522815 (+ 388us) server_negotiation.cc:407] Connection header received
0806 12:29:00.522932 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:00.522935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:00.522975 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:00.523045 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:00.523735 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45058: BlockingRecv error: recv got EOF from 127.0.0.1:45058 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:29:00.524900 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 9400
I20260806 12:29:00.533047 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 9542
I20260806 12:29:00.539084 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 9685
I20260806 12:29:00.545079 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 9300
2026-08-06T12:29:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2184 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9837](info): set up 2 sockets
Aug 06 12:29:00 dist-test-slave-1xrh krb5kdc[9837](info): commencing operation
krb5kdc: starting...
W20260806 12:29:01.619418 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:01 dist-test-slave-1xrh krb5kdc[9837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:01.742012 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:44007
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:39885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:44007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:01.851001 9852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:01.851315 9852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:01.851401 9852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:01.855293 9852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:01.855367 9852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:01.855401 9852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:01.855436 9852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:01.855469 9852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:01.860507 9852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:39885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:44007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:44007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:01.861424 9852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:01.862373 9852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:01.870662 9858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:01.870708 9857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:01.870704 9862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:01.870977 9852 server_base.cc:1034] running on GCE node
I20260806 12:29:01.871536 9852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:01.872570 9852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:01.873760 9852 hybrid_clock.cc:648] HybridClock initialized: now 1786019341873746 us; error 36 us; skew 500 ppm
Aug 06 12:29:01 dist-test-slave-1xrh krb5kdc[9837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:01.876531 9852 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:01.877488 9852 webserver.cc:460] Webserver started at http://127.6.152.190:42295/ using document root <none> and password file <none>
I20260806 12:29:01.878064 9852 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:01.878159 9852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:01.878397 9852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:01.880472 9852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "cf0910979c22412f8b31ecb0f5a0de89"
format_stamp: "Formatted at 2026-08-06 12:29:01 on dist-test-slave-1xrh"
server_key: "96753f5eb99a83f68c63567069b493ee"
server_key_iv: "f460d19a541e1047f57f63d930d766f8"
server_key_version: "encryptionkey@0"
I20260806 12:29:01.881033 9852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "cf0910979c22412f8b31ecb0f5a0de89"
format_stamp: "Formatted at 2026-08-06 12:29:01 on dist-test-slave-1xrh"
server_key: "96753f5eb99a83f68c63567069b493ee"
server_key_iv: "f460d19a541e1047f57f63d930d766f8"
server_key_version: "encryptionkey@0"
I20260806 12:29:01.884594 9852 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 12:29:01.886952 9869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:01.888234 9852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:01.888389 9852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "cf0910979c22412f8b31ecb0f5a0de89"
format_stamp: "Formatted at 2026-08-06 12:29:01 on dist-test-slave-1xrh"
server_key: "96753f5eb99a83f68c63567069b493ee"
server_key_iv: "f460d19a541e1047f57f63d930d766f8"
server_key_version: "encryptionkey@0"
I20260806 12:29:01.888502 9852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:01.905236 9852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:01.907946 9852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:01.908144 9852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:01.915064 9852 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:44007
I20260806 12:29:01.915370 9922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:44007 every 8 connection(s)
I20260806 12:29:01.916029 9852 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:01.917743 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 9852
I20260806 12:29:01.917861 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:01.918166 6754 external_mini_cluster.cc:1428] Setting key bc5f157493b0a9dca6497c5a439eb9c4
I20260806 12:29:01.919104 9923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:01.925607 9923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89: Bootstrap starting.
Aug 06 12:29:01 dist-test-slave-1xrh krb5kdc[9837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:01.928195 9923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:01.929066 9923 log.cc:824] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:01.932631 9923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89: No bootstrap required, opened a new log
I20260806 12:29:01.935036 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:01.919489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.6.152.190:44007)
0806 12:29:01.919901 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:01.919918 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:01.919952 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:29:01.920585 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:01.920600 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:01.920904 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:01.921171 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:01.922151 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:01.923106 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:01.924339 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:01.924782 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:01.924893 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:01.927643 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:01.927648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:01.927661 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:01.927688 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:01.929465 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:01.932665 (+ 3200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:01.932671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:01.932677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:01.932814 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:01.933127 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:01.933130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:01.933132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:01.933630 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:01.933776 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:01.934052 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 12:29:01.934342 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":138,"threads_started":1}
I20260806 12:29:01.936581 9923 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0910979c22412f8b31ecb0f5a0de89" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 44007 } }
I20260806 12:29:01.936846 9923 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:01.936914 9923 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0910979c22412f8b31ecb0f5a0de89, State: Initialized, Role: FOLLOWER
I20260806 12:29:01.937371 9923 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0910979c22412f8b31ecb0f5a0de89" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 44007 } }
I20260806 12:29:01.937522 9923 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:01.937592 9923 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:01.937712 9923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:01.937790 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:29:01.938606 9923 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0910979c22412f8b31ecb0f5a0de89" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 44007 } }
I20260806 12:29:01.938926 9923 leader_election.cc:302] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf0910979c22412f8b31ecb0f5a0de89; no voters:
I20260806 12:29:01.939272 9923 leader_election.cc:288] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:01.939386 9928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:01.939647 9928 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [term 1 LEADER]: Becoming Leader. State: Replica: cf0910979c22412f8b31ecb0f5a0de89, State: Running, Role: LEADER
I20260806 12:29:01.940053 9928 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0910979c22412f8b31ecb0f5a0de89" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 44007 } }
I20260806 12:29:01.940482 9923 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:01.941651 9929 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf0910979c22412f8b31ecb0f5a0de89. Latest consensus state: current_term: 1 leader_uuid: "cf0910979c22412f8b31ecb0f5a0de89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0910979c22412f8b31ecb0f5a0de89" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 44007 } } }
I20260806 12:29:01.941862 9929 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:01.942245 9930 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf0910979c22412f8b31ecb0f5a0de89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0910979c22412f8b31ecb0f5a0de89" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 44007 } } }
I20260806 12:29:01.942926 9930 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:01.942984 9939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:01.946240 9939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:01.952291 9939 catalog_manager.cc:1347] Generated new cluster ID: 7c1c5458cabd477f87e5a378188affd1
I20260806 12:29:01.952400 9939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:01.959085 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:01.953635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.6.152.190:44007)
0806 12:29:01.953753 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:01.953758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:01.953960 (+ 202us) server_negotiation.cc:407] Connection header received
0806 12:29:01.954075 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:01.954078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:01.954129 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:01.954214 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:01.954744 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:01.955593 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:01.956741 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:01.957093 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:01.957204 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:01.958099 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:01.958103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:01.958107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:01.958641 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:01.958677 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:01.958696 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:01.958789 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:01.958890 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 12:29:01.958950 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:29:01.964025 9939 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:01.965118 9939 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:01.975978 9939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf0910979c22412f8b31ecb0f5a0de89: Generated new TSK 0
I20260806 12:29:01.976750 9939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:01.990581 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 9852
2026-08-06T12:29:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1445 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:02 dist-test-slave-1xrh krb5kdc[9957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:02 dist-test-slave-1xrh krb5kdc[9957](info): set up 2 sockets
Aug 06 12:29:02 dist-test-slave-1xrh krb5kdc[9957](info): commencing operation
krb5kdc: starting...
W20260806 12:29:03.055558 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:03.177879 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:42167
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:03.288240 9970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:03.288492 9970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:03.288558 9970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:03.292536 9970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:03.292591 9970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:03.292611 9970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:03.292672 9970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:03.292729 9970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:03.298108 9970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:42167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:03.299186 9970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:03.300217 9970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:03.308920 9979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:03.308938 9977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:03.308915 9975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:03.308988 9970 server_base.cc:1034] running on GCE node
I20260806 12:29:03.309913 9970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:03.311030 9970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:03.312218 9970 hybrid_clock.cc:648] HybridClock initialized: now 1786019343312186 us; error 44 us; skew 500 ppm
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:03.315228 9970 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:03.316412 9970 webserver.cc:460] Webserver started at http://127.6.152.190:39933/ using document root <none> and password file <none>
I20260806 12:29:03.316977 9970 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:03.317081 9970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:03.317348 9970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:03.320384 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "4a41e6a981bf419cb2352a70832b1954"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "b05906a117474699b44091b67c64fb6c"
server_key_iv: "99b8f84a66afea5c8e233af2bc3fbd35"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.320968 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "4a41e6a981bf419cb2352a70832b1954"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "b05906a117474699b44091b67c64fb6c"
server_key_iv: "99b8f84a66afea5c8e233af2bc3fbd35"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.324609 9970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 12:29:03.326921 9986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.328293 9970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:03.328433 9970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "4a41e6a981bf419cb2352a70832b1954"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "b05906a117474699b44091b67c64fb6c"
server_key_iv: "99b8f84a66afea5c8e233af2bc3fbd35"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.328562 9970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:03.341357 9970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:03.344058 9970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:03.344255 9970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:03.351159 9970 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:42167
I20260806 12:29:03.351435 10042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:42167 every 8 connection(s)
I20260806 12:29:03.352110 9970 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:03.353823 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 9970
I20260806 12:29:03.353955 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:03.354264 6754 external_mini_cluster.cc:1428] Setting key 9a732c8b3d6d6cb39e6abb9c564ed146
I20260806 12:29:03.355181 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:03.361766 10043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954: Bootstrap starting.
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:03.364430 10043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:03.365165 10043 log.cc:824] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:03.367208 10043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954: No bootstrap required, opened a new log
I20260806 12:29:03.369138 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:03.355633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.6.152.190:42167)
0806 12:29:03.356039 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:03.356060 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:03.356108 (+ 48us) server_negotiation.cc:407] Connection header received
0806 12:29:03.356729 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:03.356743 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:03.357052 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:03.357323 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:03.358302 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.359409 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.360569 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.361032 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.361164 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:03.364168 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:03.364174 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:03.364187 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:03.364214 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:03.366190 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:03.366655 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:03.366661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:03.366666 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:03.366778 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:03.367087 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:03.367091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:03.367093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:03.367608 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:03.367822 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:03.368125 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 12:29:03.368427 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":135,"threads_started":1}
I20260806 12:29:03.370826 10043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a41e6a981bf419cb2352a70832b1954" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42167 } }
I20260806 12:29:03.371065 10043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:03.371169 10043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a41e6a981bf419cb2352a70832b1954, State: Initialized, Role: FOLLOWER
I20260806 12:29:03.371621 10043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a41e6a981bf419cb2352a70832b1954" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42167 } }
I20260806 12:29:03.371785 10043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:03.371863 10043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:03.371990 10043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:03.372870 10043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a41e6a981bf419cb2352a70832b1954" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42167 } }
I20260806 12:29:03.373198 10043 leader_election.cc:302] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a41e6a981bf419cb2352a70832b1954; no voters:
I20260806 12:29:03.373545 10043 leader_election.cc:288] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:03.373672 10048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:03.373932 10048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [term 1 LEADER]: Becoming Leader. State: Replica: 4a41e6a981bf419cb2352a70832b1954, State: Running, Role: LEADER
I20260806 12:29:03.374233 10048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a41e6a981bf419cb2352a70832b1954" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42167 } }
I20260806 12:29:03.374810 10043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:03.376051 10049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a41e6a981bf419cb2352a70832b1954" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a41e6a981bf419cb2352a70832b1954" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42167 } } }
I20260806 12:29:03.376056 10050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a41e6a981bf419cb2352a70832b1954. Latest consensus state: current_term: 1 leader_uuid: "4a41e6a981bf419cb2352a70832b1954" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a41e6a981bf419cb2352a70832b1954" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42167 } } }
I20260806 12:29:03.376199 10050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:03.376199 10049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:03.376695 10058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:03.379384 10058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:03.384789 10058 catalog_manager.cc:1347] Generated new cluster ID: 56d93473ca534261b76650e15e436aff
I20260806 12:29:03.384856 10058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:03.400214 10058 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:03.401193 10058 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:03.416118 10058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a41e6a981bf419cb2352a70832b1954: Generated new TSK 0
I20260806 12:29:03.416857 10058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:29:03.446921 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42167
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:03.557653 10070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:03.557920 10070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:03.558009 10070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:03.561949 10070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:03.562040 10070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:03.562134 10070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:03.567245 10070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:03.568259 10070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:03.569254 10070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:03.579191 10075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:03.579257 10076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:03.579349 10070 server_base.cc:1034] running on GCE node
W20260806 12:29:03.579396 10078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:03.579936 10070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:03.580580 10070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:03.581764 10070 hybrid_clock.cc:648] HybridClock initialized: now 1786019343581737 us; error 37 us; skew 500 ppm
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:03.589072 10070 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:29:03.589985 10070 webserver.cc:460] Webserver started at http://127.6.152.129:44127/ using document root <none> and password file <none>
I20260806 12:29:03.590560 10070 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:03.590623 10070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:03.590883 10070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:03.593086 10070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "80afac2a86d04294a94b4a74a485e253"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "cde0e165bdc5abf6e7a2c3c6c0176e93"
server_key_iv: "a020a2c690d4758ab428095c734f8901"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.593675 10070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "80afac2a86d04294a94b4a74a485e253"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "cde0e165bdc5abf6e7a2c3c6c0176e93"
server_key_iv: "a020a2c690d4758ab428095c734f8901"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.597505 10070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 12:29:03.600085 10085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.601284 10070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:03.601419 10070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "80afac2a86d04294a94b4a74a485e253"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "cde0e165bdc5abf6e7a2c3c6c0176e93"
server_key_iv: "a020a2c690d4758ab428095c734f8901"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.601536 10070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:03.613327 10070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:03.616026 10070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:03.616221 10070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:03.616781 10070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:03.617570 10070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:03.617642 10070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.617713 10070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:03.617764 10070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.626627 10070 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:40541
I20260806 12:29:03.626649 10201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:40541 every 8 connection(s)
I20260806 12:29:03.627542 10070 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:03.633548 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10070
I20260806 12:29:03.633674 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:03.634001 6754 external_mini_cluster.cc:1428] Setting key e7cacb4f97ef81dccd88e9ecea3d44b9
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:03.642079 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:03.629341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:47871 (local address 127.6.152.190:42167)
0806 12:29:03.629481 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:03.629485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:03.630237 (+ 752us) server_negotiation.cc:407] Connection header received
0806 12:29:03.631744 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:03.631747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:03.631802 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:03.631878 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:03.633066 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.633874 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.635053 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.635375 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.635489 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:03.638002 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:03.638008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:03.638011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:03.638053 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:03.639637 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:03.640279 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:03.640282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:03.640283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:03.640359 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:03.640765 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:03.640771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:03.640774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:03.641065 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:03.641185 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:03.641663 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 12:29:03.641820 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:29:03.642656 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:03.629633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:42167 (local address 127.6.152.129:47871)
0806 12:29:03.630069 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:03.630122 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:03.631528 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:03.632057 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:03.632064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:03.632403 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:03.632932 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:03.632946 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.634009 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.634012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:03.634912 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:03.634922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.635528 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.635545 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:03.635704 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:03.636292 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:03.636308 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:03.637831 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:03.639763 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:03.639772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:03.639790 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:03.640178 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:03.640461 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:03.640467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:03.640469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:03.640659 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:03.641178 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:03.641183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:03.641396 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 12:29:03.641563 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 12:29:03.641811 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260806 12:29:03.644027 10202 heartbeater.cc:347] Connected to a master server at 127.6.152.190:42167
I20260806 12:29:03.644320 10202 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:29:03.644928 10202 heartbeater.cc:511] Master 127.6.152.190:42167 requested a full tablet report, sending...
I20260806 12:29:03.646471 10004 ts_manager.cc:194] Registered new tserver with Master: 80afac2a86d04294a94b4a74a485e253 (127.6.152.129:40541)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:29:03.647789 10004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:47871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:03.662333 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42167
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:03.773108 10207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:03.773351 10207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:03.773392 10207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:03.777434 10207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:03.777518 10207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:03.777626 10207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:03.782650 10207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:03.783573 10207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:03.784524 10207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:03.793910 10213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:03.793933 10212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:03.793864 10216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:03.794095 10207 server_base.cc:1034] running on GCE node
I20260806 12:29:03.794467 10207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:03.795111 10207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:03.796339 10207 hybrid_clock.cc:648] HybridClock initialized: now 1786019343796313 us; error 46 us; skew 500 ppm
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:03.799073 10207 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:03.799966 10207 webserver.cc:460] Webserver started at http://127.6.152.130:38787/ using document root <none> and password file <none>
I20260806 12:29:03.800505 10207 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:03.800611 10207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:03.800840 10207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:03.802783 10207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "da0a1008f43e46a1ba812e9b093a1333"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "22bc83341f4229cea1346a87f0733144"
server_key_iv: "7302510a5918bcdda95dbc00dfefed91"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.803365 10207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "da0a1008f43e46a1ba812e9b093a1333"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "22bc83341f4229cea1346a87f0733144"
server_key_iv: "7302510a5918bcdda95dbc00dfefed91"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.806946 10207 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 12:29:03.809221 10223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.810366 10207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 12:29:03.810508 10207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "da0a1008f43e46a1ba812e9b093a1333"
format_stamp: "Formatted at 2026-08-06 12:29:03 on dist-test-slave-1xrh"
server_key: "22bc83341f4229cea1346a87f0733144"
server_key_iv: "7302510a5918bcdda95dbc00dfefed91"
server_key_version: "encryptionkey@0"
I20260806 12:29:03.810623 10207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:03.834205 10207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:03.837620 10207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:03.837831 10207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:03.838475 10207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:03.839519 10207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:03.839607 10207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.839689 10207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:03.839740 10207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:03.851807 10207 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:41979
I20260806 12:29:03.851836 10342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:41979 every 8 connection(s)
I20260806 12:29:03.852735 10207 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:03.858806 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10207
I20260806 12:29:03.858978 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:03.859315 6754 external_mini_cluster.cc:1428] Setting key 0896a91e356803e48b1e40adda591b6e
Aug 06 12:29:03 dist-test-slave-1xrh krb5kdc[9957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:03.867399 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:03.854540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:60045 (local address 127.6.152.190:42167)
0806 12:29:03.854711 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:03.854717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:03.855592 (+ 875us) server_negotiation.cc:407] Connection header received
0806 12:29:03.856915 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:03.856918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:03.856969 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:03.857053 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:03.858125 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.858712 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.860202 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.860450 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.860571 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:03.863331 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:03.863337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:03.863342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:03.863373 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:03.865102 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:03.865602 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:03.865609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:03.865614 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:03.865711 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:03.866115 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:03.866119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:03.866122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:03.866387 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:03.866510 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:03.866974 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 12:29:03.867155 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 12:29:03.868052 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:03.854967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:42167 (local address 127.6.152.130:60045)
0806 12:29:03.855447 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:03.855489 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:03.856700 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:03.857200 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:03.857206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:03.857534 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:03.857993 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:03.858006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.858872 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.858875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:03.860052 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:03.860063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:03.860621 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:03.860625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:03.860821 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:03.861421 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:03.861438 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:03.863081 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:03.865218 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:03.865222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:03.865233 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:03.865481 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:03.865809 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:03.865811 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:03.865813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:03.865972 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:03.866502 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:03.866507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:03.866716 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 12:29:03.866858 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 12:29:03.867107 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":166,"threads_started":1}
I20260806 12:29:03.869261 10343 heartbeater.cc:347] Connected to a master server at 127.6.152.190:42167
I20260806 12:29:03.869489 10343 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:03.869946 10343 heartbeater.cc:511] Master 127.6.152.190:42167 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
I20260806 12:29:03.870983 10004 ts_manager.cc:194] Registered new tserver with Master: da0a1008f43e46a1ba812e9b093a1333 (127.6.152.130:41979)
I20260806 12:29:03.871605 10004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:60045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:03.887980 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42167
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:03.998827 10348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:03.999130 10348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:03.999259 10348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:04.003449 10348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:04.003521 10348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:04.003603 10348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:04.008914 10348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:42167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:04.009902 10348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:04.010896 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:04.020123 10356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:04.020170 10353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:04.020123 10354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:04.020488 10348 server_base.cc:1034] running on GCE node
I20260806 12:29:04.020875 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:04.021498 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:04.022692 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786019344022667 us; error 49 us; skew 500 ppm
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019344, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:04.025326 10348 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:04.026124 10348 webserver.cc:460] Webserver started at http://127.6.152.131:37255/ using document root <none> and password file <none>
I20260806 12:29:04.026680 10348 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:04.026767 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:04.027006 10348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:04.029048 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "8ed7bf4ebcbf4ffb85264ed3b3fa1226"
format_stamp: "Formatted at 2026-08-06 12:29:04 on dist-test-slave-1xrh"
server_key: "566a062c5754216929ae95a44daf6443"
server_key_iv: "51c77e73debf7fec8ba219a77d3ce363"
server_key_version: "encryptionkey@0"
I20260806 12:29:04.029619 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "8ed7bf4ebcbf4ffb85264ed3b3fa1226"
format_stamp: "Formatted at 2026-08-06 12:29:04 on dist-test-slave-1xrh"
server_key: "566a062c5754216929ae95a44daf6443"
server_key_iv: "51c77e73debf7fec8ba219a77d3ce363"
server_key_version: "encryptionkey@0"
I20260806 12:29:04.033361 10348 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:04.035669 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:04.036841 10348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:04.036975 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "8ed7bf4ebcbf4ffb85264ed3b3fa1226"
format_stamp: "Formatted at 2026-08-06 12:29:04 on dist-test-slave-1xrh"
server_key: "566a062c5754216929ae95a44daf6443"
server_key_iv: "51c77e73debf7fec8ba219a77d3ce363"
server_key_version: "encryptionkey@0"
I20260806 12:29:04.037092 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:04.049777 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:04.052549 10348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:04.052758 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:04.053362 10348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:04.054294 10348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:04.054365 10348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:04.054432 10348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:04.054481 10348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:04.063678 10348 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:44809
I20260806 12:29:04.063704 10483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:44809 every 8 connection(s)
I20260806 12:29:04.064574 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[9957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019344, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:04.073863 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10348
I20260806 12:29:04.073993 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:04.074303 6754 external_mini_cluster.cc:1428] Setting key 7c402c067d7e0b430384bf8e67854e69
I20260806 12:29:04.078135 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:04.066277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:49157 (local address 127.6.152.190:42167)
0806 12:29:04.066444 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:04.066449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:04.067311 (+ 862us) server_negotiation.cc:407] Connection header received
0806 12:29:04.068753 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:04.068756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:04.068803 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:04.068895 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:04.070108 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:04.070603 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:04.071746 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:04.071954 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:04.072040 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:04.074536 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:04.074540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:04.074542 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:04.074604 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:04.076389 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:04.076900 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:04.076903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:04.076904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:04.076980 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:04.077254 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:04.077256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:04.077257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:04.077429 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:04.077506 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:04.077916 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 12:29:04.078015 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:29:04.078934 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:04.066655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:42167 (local address 127.6.152.131:49157)
0806 12:29:04.067101 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:04.067205 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:04.068507 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:04.069071 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:04.069077 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:04.069512 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:04.069984 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:04.069997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:04.070700 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:04.070702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:04.071616 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:04.071644 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:04.072077 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:04.072080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:04.072211 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:04.072723 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:04.072738 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:04.074330 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:04.076488 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:04.076492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:04.076503 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:04.076816 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:04.077060 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:04.077062 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:04.077064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:04.077176 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:04.077519 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:04.077523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:04.077724 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 12:29:04.077867 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 12:29:04.078111 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
I20260806 12:29:04.079998 10484 heartbeater.cc:347] Connected to a master server at 127.6.152.190:42167
I20260806 12:29:04.080220 10484 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:04.080657 10484 heartbeater.cc:511] Master 127.6.152.190:42167 requested a full tablet report, sending...
I20260806 12:29:04.081575 10004 ts_manager.cc:194] Registered new tserver with Master: 8ed7bf4ebcbf4ffb85264ed3b3fa1226 (127.6.152.131:44809)
I20260806 12:29:04.082180 10004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:49157
I20260806 12:29:04.088115 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019344, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[9957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019344, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:04.116055 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:04.107631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.6.152.190:42167)
0806 12:29:04.107793 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:04.107799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:04.107931 (+ 132us) server_negotiation.cc:407] Connection header received
0806 12:29:04.108048 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:04.108051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:04.108096 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:04.108167 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:04.108841 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:04.109351 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:04.110159 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:04.110374 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:04.110471 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:04.112841 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:04.112844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:04.112846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:04.112871 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:04.114380 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:04.114866 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:04.114869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:04.114870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:04.114947 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:04.115232 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:04.115234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:04.115235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:04.115463 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:04.115581 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:04.115753 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 12:29:04.115841 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 12:29:04.116446 10004 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45942
I20260806 12:29:04.117271 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10070
I20260806 12:29:04.124482 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10207
I20260806 12:29:04.130597 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10348
I20260806 12:29:04.137979 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 9970
2026-08-06T12:29:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2147 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[10496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[10496](info): set up 2 sockets
Aug 06 12:29:04 dist-test-slave-1xrh krb5kdc[10496](info): commencing operation
krb5kdc: starting...
W20260806 12:29:05.195076 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:05 dist-test-slave-1xrh krb5kdc[10496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:05.319478 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37773
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:05.436987 10509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:05.437224 10509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:05.437288 10509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:05.441345 10509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:05.441395 10509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:05.441413 10509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:05.441448 10509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:05.441463 10509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:05.446748 10509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:05.447783 10509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:05.448848 10509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:05.456913 10515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:05.456924 10517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:05.457106 10509 server_base.cc:1034] running on GCE node
W20260806 12:29:05.456924 10514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:05.457871 10509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:05.458932 10509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:05.460115 10509 hybrid_clock.cc:648] HybridClock initialized: now 1786019345460108 us; error 61 us; skew 500 ppm
Aug 06 12:29:05 dist-test-slave-1xrh krb5kdc[10496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:05.462824 10509 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:05.463716 10509 webserver.cc:460] Webserver started at http://127.6.152.190:44873/ using document root <none> and password file <none>
I20260806 12:29:05.464251 10509 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:05.464341 10509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:05.464583 10509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:05.466519 10509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "34c6b916ae5e4b9aa33b08a2719e1bee"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "bff0eaacd090d887516c2a709a168c12"
server_key_iv: "b21c055184dee8b08152d761b2cbd04e"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.467095 10509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "34c6b916ae5e4b9aa33b08a2719e1bee"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "bff0eaacd090d887516c2a709a168c12"
server_key_iv: "b21c055184dee8b08152d761b2cbd04e"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.470695 10509 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:29:05.473004 10524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.474195 10509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:05.474346 10509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "34c6b916ae5e4b9aa33b08a2719e1bee"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "bff0eaacd090d887516c2a709a168c12"
server_key_iv: "b21c055184dee8b08152d761b2cbd04e"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.474452 10509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:05.499070 10509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:05.502131 10509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:05.502280 10509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:05.509389 10509 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37773
I20260806 12:29:05.509481 10578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37773 every 8 connection(s)
I20260806 12:29:05.510360 10509 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:05.513015 10579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:05.517017 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10509
I20260806 12:29:05.517112 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:05.517393 6754 external_mini_cluster.cc:1428] Setting key 95dac086fabaf2ad7b46005ab03ca638
I20260806 12:29:05.518651 10579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee: Bootstrap starting.
I20260806 12:29:05.521358 10579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:05.522272 10579 log.cc:824] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:05.524240 10579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee: No bootstrap required, opened a new log
Aug 06 12:29:05 dist-test-slave-1xrh krb5kdc[10496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:05.528227 10579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37773 } }
I20260806 12:29:05.528460 10579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:05.528551 10579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c6b916ae5e4b9aa33b08a2719e1bee, State: Initialized, Role: FOLLOWER
I20260806 12:29:05.528963 10579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37773 } }
I20260806 12:29:05.529114 10579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:05.529201 10579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:05.529320 10579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:05.530478 10579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37773 } }
I20260806 12:29:05.530817 10579 leader_election.cc:302] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c6b916ae5e4b9aa33b08a2719e1bee; no voters:
I20260806 12:29:05.531212 10579 leader_election.cc:288] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:05.531317 10584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:05.531565 10584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [term 1 LEADER]: Becoming Leader. State: Replica: 34c6b916ae5e4b9aa33b08a2719e1bee, State: Running, Role: LEADER
I20260806 12:29:05.531922 10584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37773 } }
I20260806 12:29:05.532414 10579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:05.532683 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:05.518793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.6.152.190:37773)
0806 12:29:05.519234 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:05.519254 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:05.519296 (+ 42us) server_negotiation.cc:407] Connection header received
0806 12:29:05.520152 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:05.520174 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:05.520563 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:05.520898 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:05.521626 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:05.522987 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:05.523897 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:05.524568 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:05.524731 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:05.527482 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:05.527488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:05.527499 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:05.527524 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:05.529488 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:05.529949 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:05.529953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:05.529958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:05.530063 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:05.530360 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:05.530362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:05.530363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:05.530788 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:05.530950 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:05.531372 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 12:29:05.531752 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":111,"threads_started":1}
I20260806 12:29:05.533564 10585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37773 } } }
I20260806 12:29:05.533694 10585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:05.534039 10586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c6b916ae5e4b9aa33b08a2719e1bee. Latest consensus state: current_term: 1 leader_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c6b916ae5e4b9aa33b08a2719e1bee" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37773 } } }
I20260806 12:29:05.534147 10586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:05.535115 10594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:05.538034 10594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:05.543061 10594 catalog_manager.cc:1347] Generated new cluster ID: eddc4d0bf3254b5f9e89fd3fb3177859
I20260806 12:29:05.543141 10594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:05.567262 10594 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:05.568354 10594 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:05.573180 10594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c6b916ae5e4b9aa33b08a2719e1bee: Generated new TSK 0
I20260806 12:29:05.573832 10594 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:29:05.606163 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37773
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:05.714699 10606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:05.714964 10606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:05.715036 10606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:05.719017 10606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:05.719120 10606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:05.719228 10606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:05.724370 10606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:05.725340 10606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:05.726312 10606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:05.735783 10614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:05.735878 10612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:05.735898 10611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:05.736223 10606 server_base.cc:1034] running on GCE node
I20260806 12:29:05.736625 10606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:05.737227 10606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:05.738415 10606 hybrid_clock.cc:648] HybridClock initialized: now 1786019345738406 us; error 31 us; skew 500 ppm
Aug 06 12:29:05 dist-test-slave-1xrh krb5kdc[10496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:05.741178 10606 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:29:05.742074 10606 webserver.cc:460] Webserver started at http://127.6.152.129:32955/ using document root <none> and password file <none>
I20260806 12:29:05.742573 10606 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:05.742630 10606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:05.742786 10606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:05.744757 10606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "9878eeee6fab43a4b87c3dae5ae99f3d"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "27efa7188311870d6f223c4899de3ea3"
server_key_iv: "102e0921a4fa22e69cbee630faf2e429"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.745267 10606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "9878eeee6fab43a4b87c3dae5ae99f3d"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "27efa7188311870d6f223c4899de3ea3"
server_key_iv: "102e0921a4fa22e69cbee630faf2e429"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.748885 10606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:29:05.751201 10624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.752403 10606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:05.752524 10606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "9878eeee6fab43a4b87c3dae5ae99f3d"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "27efa7188311870d6f223c4899de3ea3"
server_key_iv: "102e0921a4fa22e69cbee630faf2e429"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.752635 10606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:05.767715 10606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:05.770459 10606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:05.770658 10606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:05.771260 10606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:05.772104 10606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:05.772163 10606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.772228 10606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:05.772262 10606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.782080 10606 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:34419
I20260806 12:29:05.782102 10754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:34419 every 8 connection(s)
I20260806 12:29:05.783035 10606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
Aug 06 12:29:05 dist-test-slave-1xrh krb5kdc[10496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:05.792618 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10606
I20260806 12:29:05.792762 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:05.793057 6754 external_mini_cluster.cc:1428] Setting key 0dc58d32a93bad2745081662b3f41489
I20260806 12:29:05.796885 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:05.785039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:32893 (local address 127.6.152.190:37773)
0806 12:29:05.785173 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:05.785177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:05.785987 (+ 810us) server_negotiation.cc:407] Connection header received
0806 12:29:05.787489 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:05.787492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:05.787541 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:05.787618 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:05.788712 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:05.789240 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:05.790294 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:05.790508 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:05.790580 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:05.793075 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:05.793079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:05.793081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:05.793114 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:05.794735 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:05.795336 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:05.795339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:05.795341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:05.795429 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:05.795783 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:05.795787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:05.795789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:05.795978 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:05.796091 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:05.796595 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 12:29:05.796717 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:29:05.797659 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:05.785365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37773 (local address 127.6.152.129:32893)
0806 12:29:05.785812 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:05.785856 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:05.787266 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:05.787744 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:05.787751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:05.788095 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:05.788572 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:05.788586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:05.789359 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:05.789361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:05.790188 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:05.790196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:05.790634 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:05.790637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:05.790773 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:05.791357 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:05.791373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:05.792907 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:05.794868 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:05.794874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:05.794888 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:05.795215 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:05.795523 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:05.795527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:05.795529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:05.795677 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:05.796102 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:05.796111 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:05.796359 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 12:29:05.796543 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 12:29:05.796851 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":107,"threads_started":1}
I20260806 12:29:05.799026 10755 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37773
I20260806 12:29:05.799309 10755 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:05.799791 10755 heartbeater.cc:511] Master 127.6.152.190:37773 requested a full tablet report, sending...
I20260806 12:29:05.801276 10543 ts_manager.cc:194] Registered new tserver with Master: 9878eeee6fab43a4b87c3dae5ae99f3d (127.6.152.129:34419)
I20260806 12:29:05.802386 10543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:32893
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:05.821863 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37773
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:05.929756 10760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:05.929991 10760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:05.930032 10760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:05.934336 10760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:05.934391 10760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:05.934453 10760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:05.939599 10760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:05.940479 10760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:05.941367 10760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:05.950464 10767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:05.950444 10766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:05.950441 10769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:05.950807 10760 server_base.cc:1034] running on GCE node
I20260806 12:29:05.951272 10760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:05.951858 10760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:05.953022 10760 hybrid_clock.cc:648] HybridClock initialized: now 1786019345952992 us; error 44 us; skew 500 ppm
Aug 06 12:29:05 dist-test-slave-1xrh krb5kdc[10496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:05.955900 10760 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:05.956845 10760 webserver.cc:460] Webserver started at http://127.6.152.130:43691/ using document root <none> and password file <none>
I20260806 12:29:05.957368 10760 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:05.957429 10760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:05.957585 10760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:05.959553 10760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "0ac1e8b8dd9444cfa7deace8d3b92180"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "3dc648cf08e6ea1a0dd267000e97df58"
server_key_iv: "52ef27c735bfed28cdbc3a6a6f9527db"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.960068 10760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "0ac1e8b8dd9444cfa7deace8d3b92180"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "3dc648cf08e6ea1a0dd267000e97df58"
server_key_iv: "52ef27c735bfed28cdbc3a6a6f9527db"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.963553 10760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:29:05.965874 10777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.967092 10760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:05.967239 10760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "0ac1e8b8dd9444cfa7deace8d3b92180"
format_stamp: "Formatted at 2026-08-06 12:29:05 on dist-test-slave-1xrh"
server_key: "3dc648cf08e6ea1a0dd267000e97df58"
server_key_iv: "52ef27c735bfed28cdbc3a6a6f9527db"
server_key_version: "encryptionkey@0"
I20260806 12:29:05.967343 10760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:05.983812 10760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:05.986897 10760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:05.987097 10760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:05.987706 10760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:05.988482 10760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:05.988529 10760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.988610 10760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:05.988644 10760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:05.996991 10760 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:42945
I20260806 12:29:05.996979 10895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:42945 every 8 connection(s)
I20260806 12:29:05.997834 10760 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[10496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019345, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:06.007704 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10760
I20260806 12:29:06.007843 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:06.008215 6754 external_mini_cluster.cc:1428] Setting key 17ec62e522ccc03027f84d2a24bdf572
I20260806 12:29:06.011468 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:05.999617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:36123 (local address 127.6.152.190:37773)
0806 12:29:05.999758 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:05.999762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:06.000587 (+ 825us) server_negotiation.cc:407] Connection header received
0806 12:29:06.001856 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:06.001859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:06.001905 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:06.001979 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:06.003050 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.003592 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.004820 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.005041 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.005124 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:06.007534 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:06.007538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:06.007540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:06.007565 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:06.009148 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.009745 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.009749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.009751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.009850 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.010201 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.010208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.010213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.010462 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:06.010583 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:06.011031 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 12:29:06.011218 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 12:29:06.012094 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:05.999968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37773 (local address 127.6.152.130:36123)
0806 12:29:06.000436 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:06.000477 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:06.001667 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:06.002133 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:06.002139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:06.002473 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:06.002902 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:06.002916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.003699 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.003702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:06.004704 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:06.004714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.005169 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.005172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:06.005299 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:06.005873 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:06.005889 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:06.007349 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:06.009274 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:06.009280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:06.009293 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:06.009636 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:06.009966 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:06.009968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:06.009970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:06.010100 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:06.010589 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:06.010594 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:06.010797 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 12:29:06.010944 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 12:29:06.011239 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":106,"threads_started":1}
I20260806 12:29:06.013238 10896 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37773
I20260806 12:29:06.013489 10896 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:06.013959 10896 heartbeater.cc:511] Master 127.6.152.190:37773 requested a full tablet report, sending...
I20260806 12:29:06.015007 10543 ts_manager.cc:194] Registered new tserver with Master: 0ac1e8b8dd9444cfa7deace8d3b92180 (127.6.152.130:42945)
I20260806 12:29:06.015602 10543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:36123
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:06.037897 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37773
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:06.148857 10902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:06.149098 10902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:06.149137 10902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:06.153069 10902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:06.153118 10902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:06.153169 10902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:06.158296 10902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:06.159252 10902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:06.160159 10902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:06.169363 10910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:06.169416 10907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:06.169376 10908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:06.169734 10902 server_base.cc:1034] running on GCE node
I20260806 12:29:06.170143 10902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:06.170773 10902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:06.171954 10902 hybrid_clock.cc:648] HybridClock initialized: now 1786019346171926 us; error 38 us; skew 500 ppm
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[10496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:06.174602 10902 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:06.175521 10902 webserver.cc:460] Webserver started at http://127.6.152.131:35909/ using document root <none> and password file <none>
I20260806 12:29:06.176043 10902 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:06.176132 10902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:06.176347 10902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:06.178454 10902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "5fbff75bdffd4f68b832f8a7c0c21bd4"
format_stamp: "Formatted at 2026-08-06 12:29:06 on dist-test-slave-1xrh"
server_key: "b7c3f51ba75bc6ffa66f82f6eab1ffe4"
server_key_iv: "4b96acb21cf173fcb907ff16cf7fd29a"
server_key_version: "encryptionkey@0"
I20260806 12:29:06.179065 10902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "5fbff75bdffd4f68b832f8a7c0c21bd4"
format_stamp: "Formatted at 2026-08-06 12:29:06 on dist-test-slave-1xrh"
server_key: "b7c3f51ba75bc6ffa66f82f6eab1ffe4"
server_key_iv: "4b96acb21cf173fcb907ff16cf7fd29a"
server_key_version: "encryptionkey@0"
I20260806 12:29:06.182468 10902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:29:06.184774 10918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:06.185895 10902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:06.186050 10902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "5fbff75bdffd4f68b832f8a7c0c21bd4"
format_stamp: "Formatted at 2026-08-06 12:29:06 on dist-test-slave-1xrh"
server_key: "b7c3f51ba75bc6ffa66f82f6eab1ffe4"
server_key_iv: "4b96acb21cf173fcb907ff16cf7fd29a"
server_key_version: "encryptionkey@0"
I20260806 12:29:06.186167 10902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:06.201749 10902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:06.205057 10902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:06.205255 10902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:06.205814 10902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:06.206616 10902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:06.206686 10902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:06.206753 10902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:06.206795 10902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:06.216014 10902 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:33967
I20260806 12:29:06.216040 11038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:33967 every 8 connection(s)
I20260806 12:29:06.216815 10902 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:06.223624 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 10902
I20260806 12:29:06.223758 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:06.224148 6754 external_mini_cluster.cc:1428] Setting key 9de9df318d71ecd58c45a8dcc09bd5ce
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[10496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:06.230260 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:06.218601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:47801 (local address 127.6.152.190:37773)
0806 12:29:06.218761 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:06.218766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:06.219691 (+ 925us) server_negotiation.cc:407] Connection header received
0806 12:29:06.220978 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:06.220981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:06.221027 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:06.221102 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:06.222163 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.222676 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.224005 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.224264 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.224346 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:06.227067 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:06.227070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:06.227073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:06.227096 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:06.228566 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.229002 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.229005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.229006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.229099 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.229376 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.229378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.229379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.229539 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:06.229626 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:06.230008 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 12:29:06.230115 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:29:06.230990 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:06.219021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37773 (local address 127.6.152.131:47801)
0806 12:29:06.219539 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:06.219580 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:06.220791 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:06.221256 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:06.221262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:06.221599 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:06.222040 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:06.222052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.222779 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.222781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:06.223884 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:06.223896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.224407 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.224411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:06.224591 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:06.225269 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:06.225288 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:06.226925 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:06.228682 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:06.228686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:06.228697 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:06.228920 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:06.229176 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:06.229178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:06.229179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:06.229299 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:06.229633 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:06.229638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:06.229827 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 12:29:06.229962 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 12:29:06.230224 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":142,"threads_started":1}
I20260806 12:29:06.232127 11039 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37773
I20260806 12:29:06.232347 11039 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:06.232784 11039 heartbeater.cc:511] Master 127.6.152.190:37773 requested a full tablet report, sending...
I20260806 12:29:06.233762 10543 ts_manager.cc:194] Registered new tserver with Master: 5fbff75bdffd4f68b832f8a7c0c21bd4 (127.6.152.131:33967)
I20260806 12:29:06.234392 10543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:47801
I20260806 12:29:06.237731 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[10496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[10496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:29:06.264153 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:06.254358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.6.152.129:34419)
0806 12:29:06.254672 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:06.254688 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:06.254716 (+ 28us) server_negotiation.cc:407] Connection header received
0806 12:29:06.254770 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:06.254776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:06.254974 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:06.255127 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:06.255866 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.256802 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.257710 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.258068 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.258187 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:06.260626 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:06.260635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:06.260647 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:06.260668 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:06.262312 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.262764 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.262769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.262771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.262862 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.263171 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.263174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.263175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.263414 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:06.263618 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:06.263827 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 12:29:06.263995 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[10496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:06.274936 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:06.266901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41444 (local address 127.6.152.190:37773)
0806 12:29:06.267046 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:06.267051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:06.267233 (+ 182us) server_negotiation.cc:407] Connection header received
0806 12:29:06.267380 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:06.267383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:06.267424 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:06.267512 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:06.268104 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.268607 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.269386 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:06.269602 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:06.269711 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:06.272054 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:06.272062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:06.272064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:06.272086 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:06.273261 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.273675 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.273679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.273680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.273764 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:06.274051 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:06.274054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:06.274056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:06.274266 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:06.274412 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:06.274661 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 12:29:06.274778 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 12:29:06.275414 10543 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41444
I20260806 12:29:06.275962 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10606
I20260806 12:29:06.283113 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10760
I20260806 12:29:06.289209 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10902
I20260806 12:29:06.294977 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 10509
2026-08-06T12:29:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2157 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[11052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[11052](info): set up 2 sockets
Aug 06 12:29:06 dist-test-slave-1xrh krb5kdc[11052](info): commencing operation
krb5kdc: starting...
W20260806 12:29:07.355460 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:07 dist-test-slave-1xrh krb5kdc[11052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:07.477571 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37879
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:07.586613 11068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:07.586941 11068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:07.587023 11068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:07.591068 11068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:07.591177 11068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:07.591223 11068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:07.591264 11068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:07.591298 11068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:07.596657 11068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:07.597725 11068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:07.598749 11068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:07.606930 11073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:07.607028 11077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:07.606938 11075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:07.607223 11068 server_base.cc:1034] running on GCE node
I20260806 12:29:07.607841 11068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:07.608886 11068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:07.610054 11068 hybrid_clock.cc:648] HybridClock initialized: now 1786019347610024 us; error 39 us; skew 500 ppm
Aug 06 12:29:07 dist-test-slave-1xrh krb5kdc[11052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019347, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:07.612852 11068 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:07.613562 11068 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:07.613628 11068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:07.613787 11068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:07.615839 11068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "60e2c2e3b0104fcbb720faa7a28918d5"
format_stamp: "Formatted at 2026-08-06 12:29:07 on dist-test-slave-1xrh"
server_key: "adcc26bb91f51deb773de1870256d441"
server_key_iv: "2dabb764a0e9292568b4214ed3ee931d"
server_key_version: "encryptionkey@0"
I20260806 12:29:07.616379 11068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "60e2c2e3b0104fcbb720faa7a28918d5"
format_stamp: "Formatted at 2026-08-06 12:29:07 on dist-test-slave-1xrh"
server_key: "adcc26bb91f51deb773de1870256d441"
server_key_iv: "2dabb764a0e9292568b4214ed3ee931d"
server_key_version: "encryptionkey@0"
I20260806 12:29:07.619777 11068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:29:07.622013 11083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:07.623281 11068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:07.623409 11068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "60e2c2e3b0104fcbb720faa7a28918d5"
format_stamp: "Formatted at 2026-08-06 12:29:07 on dist-test-slave-1xrh"
server_key: "adcc26bb91f51deb773de1870256d441"
server_key_iv: "2dabb764a0e9292568b4214ed3ee931d"
server_key_version: "encryptionkey@0"
I20260806 12:29:07.623504 11068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:07.641438 11068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:07.644172 11068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:07.644368 11068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:07.651146 11068 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37879
I20260806 12:29:07.651443 11140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37879 every 8 connection(s)
I20260806 12:29:07.651801 11068 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:07.653069 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11068
I20260806 12:29:07.653214 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:07.653522 6754 external_mini_cluster.cc:1428] Setting key 87e60c91bbdf37c15d17cbad287cfe6b
I20260806 12:29:07.654870 11141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:07.661237 11141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5: Bootstrap starting.
Aug 06 12:29:07 dist-test-slave-1xrh krb5kdc[11052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:07.663610 11141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:07.664402 11141 log.cc:824] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:07.666164 11141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5: No bootstrap required, opened a new log
I20260806 12:29:07.667636 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:07.654787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.6.152.190:37879)
0806 12:29:07.655245 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:07.655270 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:07.655305 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:29:07.655943 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:07.655958 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:07.656262 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:07.656526 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:07.657395 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:07.658335 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:07.659537 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:07.659987 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:07.660111 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:07.662821 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:07.662826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:07.662838 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:07.662864 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:07.664638 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:07.665093 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:07.665106 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:07.665111 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:07.665224 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:07.665507 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:07.665511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:07.665513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:07.665883 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:07.666105 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:07.666426 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 12:29:07.666718 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":190,"threads_started":1}
I20260806 12:29:07.670284 11141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37879 } }
I20260806 12:29:07.670483 11141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:07.670518 11141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e2c2e3b0104fcbb720faa7a28918d5, State: Initialized, Role: FOLLOWER
I20260806 12:29:07.670897 11141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37879 } }
I20260806 12:29:07.671017 11141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:07.671062 11141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:07.671178 11141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:07.671993 11141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37879 } }
I20260806 12:29:07.672338 11141 leader_election.cc:302] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60e2c2e3b0104fcbb720faa7a28918d5; no voters:
I20260806 12:29:07.672669 11141 leader_election.cc:288] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:07.672741 11147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:07.672947 11147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [term 1 LEADER]: Becoming Leader. State: Replica: 60e2c2e3b0104fcbb720faa7a28918d5, State: Running, Role: LEADER
I20260806 12:29:07.673224 11147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37879 } }
I20260806 12:29:07.673866 11141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:07.674857 11149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37879 } } }
I20260806 12:29:07.674887 11148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60e2c2e3b0104fcbb720faa7a28918d5. Latest consensus state: current_term: 1 leader_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60e2c2e3b0104fcbb720faa7a28918d5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37879 } } }
I20260806 12:29:07.674997 11148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:07.674995 11149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:07.675446 11156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:07.677992 11156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:07.682847 11156 catalog_manager.cc:1347] Generated new cluster ID: b5eb3e725a724b4b90909e17901c4aaf
I20260806 12:29:07.682911 11156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:07.694094 11156 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:07.694959 11156 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:07.700289 11156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5: Generated new TSK 0
I20260806 12:29:07.700932 11156 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:29:07.731436 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37879
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:07.839035 11170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:07.839310 11170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:07.839352 11170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:07.843217 11170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:07.843263 11170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:07.843318 11170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:07.848376 11170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:07.849313 11170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:07.850212 11170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:07.859290 11176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:07.859469 11170 server_base.cc:1034] running on GCE node
W20260806 12:29:07.859292 11175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:07.859506 11178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:07.859941 11170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:07.860517 11170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:07.861672 11170 hybrid_clock.cc:648] HybridClock initialized: now 1786019347861655 us; error 28 us; skew 500 ppm
Aug 06 12:29:07 dist-test-slave-1xrh krb5kdc[11052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019347, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:07.864393 11170 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:29:07.865103 11170 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:07.865162 11170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:07.865305 11170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:07.867368 11170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "a209a926f4e44365a0084e5ba9d91c1e"
format_stamp: "Formatted at 2026-08-06 12:29:07 on dist-test-slave-1xrh"
server_key: "a5cc09187bbccd7b6aaff0bb8bd96066"
server_key_iv: "f8cc2e7eb8b1ac6fe51550ba64c67aa4"
server_key_version: "encryptionkey@0"
I20260806 12:29:07.867872 11170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "a209a926f4e44365a0084e5ba9d91c1e"
format_stamp: "Formatted at 2026-08-06 12:29:07 on dist-test-slave-1xrh"
server_key: "a5cc09187bbccd7b6aaff0bb8bd96066"
server_key_iv: "f8cc2e7eb8b1ac6fe51550ba64c67aa4"
server_key_version: "encryptionkey@0"
I20260806 12:29:07.871383 11170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:29:07.873557 11184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:07.874888 11170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:07.874997 11170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "a209a926f4e44365a0084e5ba9d91c1e"
format_stamp: "Formatted at 2026-08-06 12:29:07 on dist-test-slave-1xrh"
server_key: "a5cc09187bbccd7b6aaff0bb8bd96066"
server_key_iv: "f8cc2e7eb8b1ac6fe51550ba64c67aa4"
server_key_version: "encryptionkey@0"
I20260806 12:29:07.875099 11170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:07.891249 11170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:07.894333 11170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:07.894526 11170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:07.895080 11170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:07.895890 11170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:07.895943 11170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:07.896005 11170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:07.896044 11170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:07.904471 11170 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:37547
I20260806 12:29:07.904495 11307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:37547 every 8 connection(s)
I20260806 12:29:07.904996 11170 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:07.907289 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11170
I20260806 12:29:07.907415 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:07.907765 6754 external_mini_cluster.cc:1428] Setting key 8fe623325196e7514085da91a1f34a4c
Aug 06 12:29:07 dist-test-slave-1xrh krb5kdc[11052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019347, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:29:07.918731 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:07.906634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:50267 (local address 127.6.152.190:37879)
0806 12:29:07.906768 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:07.906772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:07.907865 (+ 1093us) server_negotiation.cc:407] Connection header received
0806 12:29:07.909199 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:07.909203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:07.909264 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:07.909370 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:07.910453 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:07.911009 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:07.912146 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:07.912551 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:07.912669 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:07.915148 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:07.915155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:07.915160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:07.915212 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:07.916696 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:07.917214 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:07.917223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:07.917224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:07.917300 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:07.917649 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:07.917653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:07.917654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:07.917869 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:07.917979 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:07.918436 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 12:29:07.918570 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:29:07.919628 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:07.906960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37879 (local address 127.6.152.129:50267)
0806 12:29:07.907553 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:07.907739 (+ 186us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:07.908990 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:07.909560 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:07.909566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:07.909887 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:07.910302 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:07.910314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:07.911177 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:07.911186 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:07.912019 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:07.912028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:07.912720 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:07.912723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:07.912867 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:07.913485 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:07.913501 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:07.914955 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:07.916830 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:07.916835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:07.916848 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:07.917101 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:07.917394 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:07.917399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:07.917401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:07.917548 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:07.917980 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:07.917987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:07.918210 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 12:29:07.918392 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 12:29:07.918737 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:29:07.920903 11308 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37879
I20260806 12:29:07.921133 11308 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:07.921552 11308 heartbeater.cc:511] Master 127.6.152.190:37879 requested a full tablet report, sending...
I20260806 12:29:07.923020 11102 ts_manager.cc:194] Registered new tserver with Master: a209a926f4e44365a0084e5ba9d91c1e (127.6.152.129:37547)
I20260806 12:29:07.924248 11102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:50267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:07.936240 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37879
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:08.042201 11313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:08.042467 11313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:08.042544 11313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:08.046531 11313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:08.046613 11313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:08.046698 11313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:08.052254 11313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:08.053198 11313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:08.054141 11313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:08.063263 11319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:08.063300 11318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:08.063349 11321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:08.063328 11313 server_base.cc:1034] running on GCE node
I20260806 12:29:08.063917 11313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:08.064512 11313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:08.065675 11313 hybrid_clock.cc:648] HybridClock initialized: now 1786019348065646 us; error 41 us; skew 500 ppm
Aug 06 12:29:08 dist-test-slave-1xrh krb5kdc[11052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019348, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:08.068370 11313 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:08.069047 11313 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:08.069106 11313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:08.069258 11313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:08.071115 11313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "f113477d807e4af3b3474bfabfad2260"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "59d9a2b4ce82a87ce507985fd5f47d57"
server_key_iv: "eddf28532465be049307617f67064599"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.071797 11313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "f113477d807e4af3b3474bfabfad2260"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "59d9a2b4ce82a87ce507985fd5f47d57"
server_key_iv: "eddf28532465be049307617f67064599"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.075372 11313 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:29:08.077791 11327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.079058 11313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:08.079214 11313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "f113477d807e4af3b3474bfabfad2260"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "59d9a2b4ce82a87ce507985fd5f47d57"
server_key_iv: "eddf28532465be049307617f67064599"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.079320 11313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:08.089979 11313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:08.092705 11313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:08.092888 11313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:08.093438 11313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:08.094245 11313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:08.094316 11313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.094383 11313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:08.094439 11313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.103577 11313 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:33405
I20260806 12:29:08.103609 11450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:33405 every 8 connection(s)
I20260806 12:29:08.104146 11313 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:08.112114 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11313
I20260806 12:29:08.112274 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:08.112586 6754 external_mini_cluster.cc:1428] Setting key 73f3889ee4a88256cf2db275ffde577d
Aug 06 12:29:08 dist-test-slave-1xrh krb5kdc[11052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019348, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:08.118574 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.105848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:39927 (local address 127.6.152.190:37879)
0806 12:29:08.105982 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.105986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.106765 (+ 779us) server_negotiation.cc:407] Connection header received
0806 12:29:08.108221 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.108224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.108273 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.108347 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:08.109429 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.109952 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.111010 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.111288 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.111372 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.114013 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:08.114016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:08.114018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:08.114046 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:08.115840 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:08.116589 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:08.116595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:08.116599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:08.116709 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:08.117146 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:08.117152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:08.117155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:08.117422 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:08.117547 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.118261 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.118394 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:29:08.119596 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.106214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37879 (local address 127.6.152.130:39927)
0806 12:29:08.106619 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.106658 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.107940 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.108524 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.108531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.108850 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:08.109301 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.109314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.110060 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.110063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.110891 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.110899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.111430 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.111434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.111577 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.112148 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:08.112163 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:08.113809 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:08.115970 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:08.115976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:08.116009 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:08.116482 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:08.116832 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:08.116835 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:08.116837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:08.117029 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:08.117555 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:08.117564 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:08.117866 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.118173 (+ 307us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.118500 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":103,"threads_started":1}
I20260806 12:29:08.120922 11451 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37879
I20260806 12:29:08.121161 11451 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:08.121667 11451 heartbeater.cc:511] Master 127.6.152.190:37879 requested a full tablet report, sending...
I20260806 12:29:08.122781 11102 ts_manager.cc:194] Registered new tserver with Master: f113477d807e4af3b3474bfabfad2260 (127.6.152.130:33405)
I20260806 12:29:08.123656 11102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:39927
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:08.142334 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37879
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:08.251955 11456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:08.252190 11456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:08.252233 11456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:08.256038 11456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:08.256083 11456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:08.256176 11456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:08.261089 11456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:08.262065 11456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:08.263007 11456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:08.272233 11462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:08.272295 11461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:08.272295 11464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:08.272428 11456 server_base.cc:1034] running on GCE node
I20260806 12:29:08.272869 11456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:08.273514 11456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:08.274689 11456 hybrid_clock.cc:648] HybridClock initialized: now 1786019348274675 us; error 33 us; skew 500 ppm
Aug 06 12:29:08 dist-test-slave-1xrh krb5kdc[11052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019348, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:08.277539 11456 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:08.278287 11456 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:08.278376 11456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:08.278580 11456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:08.280551 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "5a81fed2f0fa4e61868116322b0f76c5"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "9863ab985e341dc813bc890bb1b45cd4"
server_key_iv: "8e43f184d3d91122506ae0fdb0354221"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.281090 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "5a81fed2f0fa4e61868116322b0f76c5"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "9863ab985e341dc813bc890bb1b45cd4"
server_key_iv: "8e43f184d3d91122506ae0fdb0354221"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.284727 11456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 12:29:08.286919 11470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.288059 11456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:08.288197 11456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "5a81fed2f0fa4e61868116322b0f76c5"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "9863ab985e341dc813bc890bb1b45cd4"
server_key_iv: "8e43f184d3d91122506ae0fdb0354221"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.288300 11456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:08.299275 11456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:08.302239 11456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:08.302428 11456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:08.302934 11456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:08.303742 11456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:08.303830 11456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.303915 11456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:08.303970 11456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.312505 11456 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:33527
I20260806 12:29:08.312579 11589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:33527 every 8 connection(s)
I20260806 12:29:08.313071 11456 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:08.318049 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11456
I20260806 12:29:08.318166 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:08.318449 6754 external_mini_cluster.cc:1428] Setting key b24981b2741e37e23996a3219b9e76fe
Aug 06 12:29:08 dist-test-slave-1xrh krb5kdc[11052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019348, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:08.325436 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.314708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:50561 (local address 127.6.152.190:37879)
0806 12:29:08.314846 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.314850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.315579 (+ 729us) server_negotiation.cc:407] Connection header received
0806 12:29:08.316745 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.316748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.316800 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.316877 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:08.317960 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.318634 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.319882 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.320121 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.320236 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.322452 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:08.322456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:08.322458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:08.322481 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:08.323746 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:08.324198 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:08.324201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:08.324202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:08.324276 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:08.324564 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:08.324566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:08.324567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:08.324730 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:08.324805 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.325222 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.325314 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:08.326156 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.314973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37879 (local address 127.6.152.131:50561)
0806 12:29:08.315414 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.315454 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.316551 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.317039 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.317045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.317358 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:08.317786 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.317818 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.318760 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.318764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.319750 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.319760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.320270 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.320274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.320438 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.320917 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:08.320933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:08.322294 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:08.323853 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:08.323858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:08.323868 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:08.324110 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:08.324354 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:08.324356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:08.324358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:08.324478 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:08.324816 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:08.324820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:08.325027 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.325173 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.325391 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
I20260806 12:29:08.327378 11590 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37879
I20260806 12:29:08.327656 11590 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:08.328074 11590 heartbeater.cc:511] Master 127.6.152.190:37879 requested a full tablet report, sending...
I20260806 12:29:08.329054 11102 ts_manager.cc:194] Registered new tserver with Master: 5a81fed2f0fa4e61868116322b0f76c5 (127.6.152.131:33527)
I20260806 12:29:08.329571 11102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:50561
I20260806 12:29:08.332139 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:29:08.340397 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.334018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.6.152.190:37879)
0806 12:29:08.334151 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.334156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.334273 (+ 117us) server_negotiation.cc:407] Connection header received
0806 12:29:08.334363 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.334365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.334400 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.334463 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:08.335058 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.335579 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.336342 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.336549 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.336622 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.337761 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:08.337763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:08.337765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:08.337785 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:08.338927 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:08.339351 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:08.339353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:08.339354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:08.339453 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:08.339670 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:08.339672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:08.339673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:08.339847 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:08.339947 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.340126 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.340211 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:29:08.343026 11102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:08.346081 11102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:08.361586 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.354943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37547 (local address 127.0.0.1:46624)
0806 12:29:08.355252 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.355266 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.355357 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.356399 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.356406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.356431 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:08.356610 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.356620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.358316 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.358320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.360079 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.360085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.361018 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.361022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.361133 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.361182 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.361286 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.361425 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":153,"thread_start_us":108,"threads_started":1}
I20260806 12:29:08.362031 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.354610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.6.152.130:33405)
0806 12:29:08.355798 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.355817 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.355842 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:29:08.355925 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.355950 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.356164 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.356319 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:08.358350 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.359382 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.360534 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.361139 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.361263 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.361395 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.361512 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.361604 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1081,"thread_start_us":102,"threads_started":1}
I20260806 12:29:08.362051 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.355403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.6.152.129:37547)
0806 12:29:08.355878 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.355891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.355912 (+ 21us) server_negotiation.cc:407] Connection header received
0806 12:29:08.355991 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.355998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.356217 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.356776 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:08.356961 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.358163 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.360214 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.360892 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.360981 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.361097 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.361801 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.361893 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":125,"threads_started":1}
I20260806 12:29:08.362736 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.354418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:33405 (local address 127.0.0.1:39414)
0806 12:29:08.354779 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.354830 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.354986 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.356523 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.356526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.356563 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:08.356798 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.356806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.359510 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.359516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.360402 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.360414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.361268 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.361272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.361361 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.361385 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.361463 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.361540 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":81,"thread_start_us":92,"threads_started":1}
I20260806 12:29:08.362735 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.355398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:33527 (local address 127.0.0.1:58622)
0806 12:29:08.355647 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.355662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.355754 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.356523 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.356526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.356575 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:08.356798 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.356806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.358316 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.358320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.360072 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.360085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.362319 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.362322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.362415 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.362449 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.362526 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.362600 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":189,"spinlock_wait_cycles":86272,"thread_start_us":109,"threads_started":1}
I20260806 12:29:08.363058 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.355421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.6.152.131:33527)
0806 12:29:08.356128 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.356139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.356159 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:29:08.356226 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.356232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.356399 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.356540 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:08.356939 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.358163 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.361149 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.361995 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.362120 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.362258 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.362558 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.362649 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":112,"threads_started":1}
I20260806 12:29:08.365118 11236 tablet_service.cc:1467] Processing CreateTablet for tablet dbec095cac2444f290e6999b91a1aea4 (DEFAULT_TABLE table=test-table [id=a358a7ede6c74c0894486aadcfada4f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:08.365334 11520 tablet_service.cc:1467] Processing CreateTablet for tablet dbec095cac2444f290e6999b91a1aea4 (DEFAULT_TABLE table=test-table [id=a358a7ede6c74c0894486aadcfada4f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:08.365437 11378 tablet_service.cc:1467] Processing CreateTablet for tablet dbec095cac2444f290e6999b91a1aea4 (DEFAULT_TABLE table=test-table [id=a358a7ede6c74c0894486aadcfada4f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:08.366281 11378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbec095cac2444f290e6999b91a1aea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:08.366282 11236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbec095cac2444f290e6999b91a1aea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:08.366299 11520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbec095cac2444f290e6999b91a1aea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:08.372148 11608 tablet_bootstrap.cc:492] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Bootstrap starting.
I20260806 12:29:08.373685 11609 tablet_bootstrap.cc:492] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: Bootstrap starting.
I20260806 12:29:08.374878 11608 tablet_bootstrap.cc:654] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:08.375334 11610 tablet_bootstrap.cc:492] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Bootstrap starting.
I20260806 12:29:08.375797 11609 tablet_bootstrap.cc:654] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:08.376257 11608 log.cc:824] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:08.377000 11609 log.cc:824] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:08.377274 11610 tablet_bootstrap.cc:654] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:08.378489 11608 tablet_bootstrap.cc:492] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: No bootstrap required, opened a new log
I20260806 12:29:08.378456 11610 log.cc:824] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:08.378676 11608 ts_tablet_manager.cc:1397] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 12:29:08.378847 11609 tablet_bootstrap.cc:492] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: No bootstrap required, opened a new log
I20260806 12:29:08.379040 11609 ts_tablet_manager.cc:1397] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 12:29:08.380378 11610 tablet_bootstrap.cc:492] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: No bootstrap required, opened a new log
I20260806 12:29:08.380563 11610 ts_tablet_manager.cc:1397] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:29:08.382074 11609 raft_consensus.cc:348] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.382313 11609 raft_consensus.cc:374] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:08.382367 11609 raft_consensus.cc:729] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a209a926f4e44365a0084e5ba9d91c1e, State: Initialized, Role: FOLLOWER
I20260806 12:29:08.382463 11608 raft_consensus.cc:348] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.382702 11608 raft_consensus.cc:374] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:08.382769 11608 raft_consensus.cc:729] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a81fed2f0fa4e61868116322b0f76c5, State: Initialized, Role: FOLLOWER
I20260806 12:29:08.382898 11609 consensus_queue.cc:260] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.383579 11308 heartbeater.cc:503] Master 127.6.152.190:37879 was elected leader, sending a full tablet report...
I20260806 12:29:08.384003 11609 ts_tablet_manager.cc:1428] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 12:29:08.384002 11608 consensus_queue.cc:260] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.384615 11590 heartbeater.cc:503] Master 127.6.152.190:37879 was elected leader, sending a full tablet report...
I20260806 12:29:08.384922 11608 ts_tablet_manager.cc:1428] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:29:08.385390 11610 raft_consensus.cc:348] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.385669 11610 raft_consensus.cc:374] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:08.385738 11610 raft_consensus.cc:729] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f113477d807e4af3b3474bfabfad2260, State: Initialized, Role: FOLLOWER
I20260806 12:29:08.386173 11610 consensus_queue.cc:260] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.386809 11451 heartbeater.cc:503] Master 127.6.152.190:37879 was elected leader, sending a full tablet report...
I20260806 12:29:08.387101 11610 ts_tablet_manager.cc:1428] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 12:29:08.406006 11309 tablet.cc:2369] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:29:08.414330 11614 raft_consensus.cc:482] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:08.414544 11614 raft_consensus.cc:504] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.415730 11614 leader_election.cc:288] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f113477d807e4af3b3474bfabfad2260 (127.6.152.130:33405), 5a81fed2f0fa4e61868116322b0f76c5 (127.6.152.131:33527)
I20260806 12:29:08.419909 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.416129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:52191 (local address 127.6.152.130:33405)
0806 12:29:08.416265 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.416269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.416283 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:08.416426 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.416429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.416482 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.416574 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:08.416854 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.417359 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.418748 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.419369 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.419502 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.419553 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.419760 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.419812 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:29:08.419947 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.416020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:33405 (local address 127.6.152.129:52191)
0806 12:29:08.416194 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.416211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.416307 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.416595 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.416597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.416616 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:08.416737 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.416743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.417514 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.417517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.418620 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.418630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.419502 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.419506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.419628 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.419655 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.419737 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.419806 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 12:29:08.420665 11617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.416364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:33527 (local address 127.6.152.129:38225)
0806 12:29:08.416706 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.416720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.416849 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.417111 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.417113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.417130 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:08.417251 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.417256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.418277 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.418280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.419369 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:08.419379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.420340 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.420346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:08.420428 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.420447 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.420512 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.420563 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":129,"threads_started":1}
I20260806 12:29:08.420583 11401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbec095cac2444f290e6999b91a1aea4" candidate_uuid: "a209a926f4e44365a0084e5ba9d91c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f113477d807e4af3b3474bfabfad2260" is_pre_election: true
I20260806 12:29:08.420914 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.416506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:38225 (local address 127.6.152.131:33527)
0806 12:29:08.416707 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.416712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.416808 (+ 96us) server_negotiation.cc:407] Connection header received
0806 12:29:08.416954 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.416958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.417008 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.417097 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:08.417357 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.418161 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.419491 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.420100 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.420218 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.420268 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.420569 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.420655 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 12:29:08.420922 11401 raft_consensus.cc:2454] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a209a926f4e44365a0084e5ba9d91c1e in term 0.
I20260806 12:29:08.421407 11186 leader_election.cc:302] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a209a926f4e44365a0084e5ba9d91c1e, f113477d807e4af3b3474bfabfad2260; no voters:
I20260806 12:29:08.421450 11542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbec095cac2444f290e6999b91a1aea4" candidate_uuid: "a209a926f4e44365a0084e5ba9d91c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a81fed2f0fa4e61868116322b0f76c5" is_pre_election: true
I20260806 12:29:08.421701 11614 raft_consensus.cc:2781] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:08.421768 11614 raft_consensus.cc:482] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:08.421736 11542 raft_consensus.cc:2454] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a209a926f4e44365a0084e5ba9d91c1e in term 0.
I20260806 12:29:08.421819 11614 raft_consensus.cc:3037] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:08.422982 11614 raft_consensus.cc:504] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.423374 11614 leader_election.cc:288] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [CANDIDATE]: Term 1 election: Requested vote from peers f113477d807e4af3b3474bfabfad2260 (127.6.152.130:33405), 5a81fed2f0fa4e61868116322b0f76c5 (127.6.152.131:33527)
I20260806 12:29:08.423730 11542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbec095cac2444f290e6999b91a1aea4" candidate_uuid: "a209a926f4e44365a0084e5ba9d91c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a81fed2f0fa4e61868116322b0f76c5"
I20260806 12:29:08.423730 11401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbec095cac2444f290e6999b91a1aea4" candidate_uuid: "a209a926f4e44365a0084e5ba9d91c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f113477d807e4af3b3474bfabfad2260"
I20260806 12:29:08.423877 11401 raft_consensus.cc:3037] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:08.423868 11542 raft_consensus.cc:3037] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:08.425160 11542 raft_consensus.cc:2454] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a209a926f4e44365a0084e5ba9d91c1e in term 1.
I20260806 12:29:08.425160 11401 raft_consensus.cc:2454] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a209a926f4e44365a0084e5ba9d91c1e in term 1.
I20260806 12:29:08.425467 11186 leader_election.cc:302] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a209a926f4e44365a0084e5ba9d91c1e, f113477d807e4af3b3474bfabfad2260; no voters:
I20260806 12:29:08.425628 11614 raft_consensus.cc:2781] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:08.425868 11614 raft_consensus.cc:686] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 1 LEADER]: Becoming Leader. State: Replica: a209a926f4e44365a0084e5ba9d91c1e, State: Running, Role: LEADER
I20260806 12:29:08.426201 11614 consensus_queue.cc:237] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } }
I20260806 12:29:08.429354 11099 catalog_manager.cc:5495] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e reported cstate change: term changed from 0 to 1, leader changed from <none> to a209a926f4e44365a0084e5ba9d91c1e (127.6.152.129). New cstate: current_term: 1 leader_uuid: "a209a926f4e44365a0084e5ba9d91c1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a209a926f4e44365a0084e5ba9d91c1e" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260806 12:29:08.457420 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.454285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.6.152.129:37547)
0806 12:29:08.454404 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.454407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.454420 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:08.454552 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.454555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.454607 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.454708 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:08.454932 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.455420 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.456193 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:08.456386 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:08.456460 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:08.456739 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:08.457122 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:08.457209 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.457273 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.457319 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:29:08.462950 11401 raft_consensus.cc:1261] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 1 FOLLOWER]: Refusing update from remote peer a209a926f4e44365a0084e5ba9d91c1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:08.462949 11542 raft_consensus.cc:1261] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 1 FOLLOWER]: Refusing update from remote peer a209a926f4e44365a0084e5ba9d91c1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:08.463676 11614 consensus_queue.cc:1035] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f113477d807e4af3b3474bfabfad2260" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:08.463869 11618 consensus_queue.cc:1035] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a81fed2f0fa4e61868116322b0f76c5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:08.471321 11621 mvcc.cc:205] Tried to move back new op lower bound from 7315535251297562624 to 7315535251156635648. Current Snapshot: MvccSnapshot[applied={T|T < 7315535251297562624}]
I20260806 12:29:08.471926 11623 mvcc.cc:205] Tried to move back new op lower bound from 7315535251297562624 to 7315535251156635648. Current Snapshot: MvccSnapshot[applied={T|T < 7315535251297562624}]
I20260806 12:29:08.479167 11102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:08.479344 11102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:08.481745 11102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60e2c2e3b0104fcbb720faa7a28918d5: Sending DeleteTablet for 3 replicas of tablet dbec095cac2444f290e6999b91a1aea4
I20260806 12:29:08.482502 11236 tablet_service.cc:1514] Processing DeleteTablet for tablet dbec095cac2444f290e6999b91a1aea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:08 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:46624
I20260806 12:29:08.482524 11520 tablet_service.cc:1514] Processing DeleteTablet for tablet dbec095cac2444f290e6999b91a1aea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:08 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:58622
I20260806 12:29:08.482993 11630 tablet_replica.cc:330] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e: stopping tablet replica
I20260806 12:29:08.483011 11378 tablet_service.cc:1514] Processing DeleteTablet for tablet dbec095cac2444f290e6999b91a1aea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:08 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:39414
I20260806 12:29:08.483493 11630 raft_consensus.cc:2229] T dbec095cac2444f290e6999b91a1aea4 P a209a926f4e44365a0084e5ba9d91c1e [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:08.483603 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11170
I20260806 12:29:08.484509 11632 tablet_replica.cc:330] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: stopping tablet replica
I20260806 12:29:08.484778 11632 raft_consensus.cc:2229] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:08.485051 11632 raft_consensus.cc:2258] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:08.483462 11631 tablet_replica.cc:330] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: stopping tablet replica
I20260806 12:29:08.485492 11631 raft_consensus.cc:2229] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:08.485797 11631 raft_consensus.cc:2258] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:08.486891 11632 ts_tablet_manager.cc:1905] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:08.487355 11631 ts_tablet_manager.cc:1905] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:08.489564 11631 ts_tablet_manager.cc:1918] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:08.489678 11631 log.cc:1196] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/dbec095cac2444f290e6999b91a1aea4
I20260806 12:29:08.489988 11631 ts_tablet_manager.cc:1939] T dbec095cac2444f290e6999b91a1aea4 P 5a81fed2f0fa4e61868116322b0f76c5: Deleting consensus metadata
I20260806 12:29:08.490484 11632 ts_tablet_manager.cc:1918] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:08.490608 11632 log.cc:1196] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/dbec095cac2444f290e6999b91a1aea4
I20260806 12:29:08.491003 11632 ts_tablet_manager.cc:1939] T dbec095cac2444f290e6999b91a1aea4 P f113477d807e4af3b3474bfabfad2260: Deleting consensus metadata
I20260806 12:29:08.491194 11085 catalog_manager.cc:4841] TS 5a81fed2f0fa4e61868116322b0f76c5 (127.6.152.131:33527): tablet dbec095cac2444f290e6999b91a1aea4 (table test-table [id=a358a7ede6c74c0894486aadcfada4f8]) successfully deleted
I20260806 12:29:08.492238 11085 catalog_manager.cc:4841] TS f113477d807e4af3b3474bfabfad2260 (127.6.152.130:33405): tablet dbec095cac2444f290e6999b91a1aea4 (table test-table [id=a358a7ede6c74c0894486aadcfada4f8]) successfully deleted
I20260806 12:29:08.494071 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11313
W20260806 12:29:08.494102 11087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:37547 (error 108)
I20260806 12:29:08.496639 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.496383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37547 (local address 127.0.0.1:46644)
0806 12:29:08.496512 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.496571 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:37547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 12:29:08.496958 11087 catalog_manager.cc:4584] TS a209a926f4e44365a0084e5ba9d91c1e (127.6.152.129:37547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbec095cac2444f290e6999b91a1aea4: Network error: Client connection negotiation failed: client connection to 127.6.152.129:37547: connect: Connection refused (error 111)
I20260806 12:29:08.500553 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11456
I20260806 12:29:08.506397 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11068
2026-08-06T12:29:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2211 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T12:29:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:08Z Disabled control of system clock
I20260806 12:29:08.537636 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:36429
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:36429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:29:08.645596 11637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:08.645833 11637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:08.645869 11637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:08.649648 11637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:08.649693 11637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:08.649708 11637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:08.649724 11637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:08.649737 11637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:08.654296 11637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:36429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:36429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:08.655217 11637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:08.656447 11637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:08.665105 11643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:08.665166 11637 server_base.cc:1034] running on GCE node
W20260806 12:29:08.665102 11642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:08.665177 11645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:08.666011 11637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:08.667172 11637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:08.668368 11637 hybrid_clock.cc:648] HybridClock initialized: now 1786019348668346 us; error 34 us; skew 500 ppm
I20260806 12:29:08.673648 11637 webserver.cc:460] Webserver started at http://127.6.152.190:44103/ using document root <none> and password file <none>
I20260806 12:29:08.674198 11637 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:08.674285 11637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:08.674510 11637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:08.676589 11637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "33f743a4705941aca8ac40cdbf774cbb"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "f9ace2615c953ea27f8dbc3c47a1f03d"
server_key_iv: "e39a68370f6ee88072fcba1557fc73ec"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.677182 11637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "33f743a4705941aca8ac40cdbf774cbb"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "f9ace2615c953ea27f8dbc3c47a1f03d"
server_key_iv: "e39a68370f6ee88072fcba1557fc73ec"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.680701 11637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:08.682986 11651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.684247 11637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:08.684394 11637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "33f743a4705941aca8ac40cdbf774cbb"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "f9ace2615c953ea27f8dbc3c47a1f03d"
server_key_iv: "e39a68370f6ee88072fcba1557fc73ec"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.684504 11637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:08.688933 11637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:08.689505 11637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:08.689692 11637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:08.696326 11637 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:36429
I20260806 12:29:08.696332 11707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:36429 every 8 connection(s)
I20260806 12:29:08.697142 11637 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:08.700315 11708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:08.702790 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11637
I20260806 12:29:08.702934 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:08.703270 6754 external_mini_cluster.cc:1428] Setting key d386c84b76bf148855a796166d8bda17
I20260806 12:29:08.706194 11708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb: Bootstrap starting.
I20260806 12:29:08.708743 11708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:08.709093 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.704664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.6.152.190:36429)
0806 12:29:08.705335 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.705356 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.705398 (+ 42us) server_negotiation.cc:407] Connection header received
0806 12:29:08.706237 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.706260 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.706665 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.707005 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:08.707714 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:08.707720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:08.707732 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:08.707742 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:08.707798 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:08.707817 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:08.707956 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.708016 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.708236 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260806 12:29:08.709615 11708 log.cc:824] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:08.711565 11708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb: No bootstrap required, opened a new log
I20260806 12:29:08.715335 11708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f743a4705941aca8ac40cdbf774cbb" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36429 } }
I20260806 12:29:08.715535 11708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:08.715583 11708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f743a4705941aca8ac40cdbf774cbb, State: Initialized, Role: FOLLOWER
I20260806 12:29:08.715937 11708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f743a4705941aca8ac40cdbf774cbb" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36429 } }
I20260806 12:29:08.716089 11708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:08.716136 11708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:08.716207 11708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:08.716970 11708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f743a4705941aca8ac40cdbf774cbb" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36429 } }
I20260806 12:29:08.717314 11708 leader_election.cc:302] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33f743a4705941aca8ac40cdbf774cbb; no voters:
I20260806 12:29:08.717655 11708 leader_election.cc:288] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:08.717795 11713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:08.718061 11713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [term 1 LEADER]: Becoming Leader. State: Replica: 33f743a4705941aca8ac40cdbf774cbb, State: Running, Role: LEADER
I20260806 12:29:08.718353 11713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f743a4705941aca8ac40cdbf774cbb" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36429 } }
I20260806 12:29:08.718776 11708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:08.720044 11716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33f743a4705941aca8ac40cdbf774cbb. Latest consensus state: current_term: 1 leader_uuid: "33f743a4705941aca8ac40cdbf774cbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f743a4705941aca8ac40cdbf774cbb" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36429 } } }
I20260806 12:29:08.720075 11715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33f743a4705941aca8ac40cdbf774cbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f743a4705941aca8ac40cdbf774cbb" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36429 } } }
I20260806 12:29:08.720189 11716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:08.720198 11715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:08.720607 11723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:08.723096 11723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:08.727846 11723 catalog_manager.cc:1347] Generated new cluster ID: f32b06b7f8514b0b96d1b0b28d4c0c3c
I20260806 12:29:08.727910 11723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:08.739816 11723 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:08.740145 11723 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:08.752027 11723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb: Generated new TSK 0
I20260806 12:29:08.752854 11723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:08.762486 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36429
--builtin_ntp_servers=127.6.152.148:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:29:08.871662 11733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:08.871883 11733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:08.871920 11733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:08.875744 11733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:08.875788 11733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:08.875840 11733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:08.880327 11733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:08.881188 11733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:08.882404 11733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:08.891651 11739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:08.891762 11733 server_base.cc:1034] running on GCE node
W20260806 12:29:08.891685 11742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:08.891655 11740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:08.892274 11733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:08.892868 11733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:08.894060 11733 hybrid_clock.cc:648] HybridClock initialized: now 1786019348894036 us; error 36 us; skew 500 ppm
I20260806 12:29:08.895828 11733 webserver.cc:460] Webserver started at http://127.6.152.129:35459/ using document root <none> and password file <none>
I20260806 12:29:08.896368 11733 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:08.896461 11733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:08.896684 11733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:08.898607 11733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "497d9a0be56d4abd93f6649fb70121c3"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "01b2626c6cdb0ad522c124dfc393d527"
server_key_iv: "e65f786f128c3373ac8e9c987ca6fe94"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.899206 11733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "497d9a0be56d4abd93f6649fb70121c3"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "01b2626c6cdb0ad522c124dfc393d527"
server_key_iv: "e65f786f128c3373ac8e9c987ca6fe94"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.902673 11733 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 12:29:08.904924 11749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.906066 11733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 12:29:08.906209 11733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "497d9a0be56d4abd93f6649fb70121c3"
format_stamp: "Formatted at 2026-08-06 12:29:08 on dist-test-slave-1xrh"
server_key: "01b2626c6cdb0ad522c124dfc393d527"
server_key_iv: "e65f786f128c3373ac8e9c987ca6fe94"
server_key_version: "encryptionkey@0"
I20260806 12:29:08.906318 11733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:08.910831 11733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:08.911393 11733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:08.911572 11733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:08.912108 11733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:08.912952 11733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:08.913024 11733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.913103 11733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:08.913139 11733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:08.921859 11733 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:34459
I20260806 12:29:08.921908 11867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:34459 every 8 connection(s)
I20260806 12:29:08.922680 11733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:08.928702 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.924324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:59731 (local address 127.6.152.190:36429)
0806 12:29:08.924458 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:08.924462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:08.925515 (+ 1053us) server_negotiation.cc:407] Connection header received
0806 12:29:08.926997 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:08.927000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:08.927039 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:08.927114 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:08.928276 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:08.928280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:08.928282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:08.928294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:08.928313 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:08.928321 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:08.928401 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:08.928557 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 12:29:08.928595 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:29:08.928843 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11733
I20260806 12:29:08.928980 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:08.929314 6754 external_mini_cluster.cc:1428] Setting key 2b98484646f120ff08eb0ef5e9b9ff0d
I20260806 12:29:08.929941 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:08.924624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:36429 (local address 127.6.152.129:59731)
0806 12:29:08.925352 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:08.925395 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:08.926748 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:08.927358 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:08.927366 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:08.927455 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:08.927984 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:08.927986 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:08.928026 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:08.928095 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:08.928409 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:08.928417 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:08.928421 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:29:08.928580 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 12:29:08.928865 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
I20260806 12:29:08.931440 11868 heartbeater.cc:347] Connected to a master server at 127.6.152.190:36429
I20260806 12:29:08.931676 11868 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:08.931713 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36429
--builtin_ntp_servers=127.6.152.148:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 12:29:08.932132 11868 heartbeater.cc:511] Master 127.6.152.190:36429 requested a full tablet report, sending...
I20260806 12:29:08.933617 11669 ts_manager.cc:194] Registered new tserver with Master: 497d9a0be56d4abd93f6649fb70121c3 (127.6.152.129:34459)
W20260806 12:29:09.039438 11871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:09.039680 11871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:09.039731 11871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:09.043610 11871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:09.043653 11871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:09.043738 11871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:09.048353 11871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:09.049336 11871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:09.050645 11871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:09.059863 11879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:09.059955 11871 server_base.cc:1034] running on GCE node
W20260806 12:29:09.059880 11877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:09.059829 11876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:09.060465 11871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:09.061162 11871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:09.062325 11871 hybrid_clock.cc:648] HybridClock initialized: now 1786019349062304 us; error 32 us; skew 500 ppm
I20260806 12:29:09.064116 11871 webserver.cc:460] Webserver started at http://127.6.152.130:41297/ using document root <none> and password file <none>
I20260806 12:29:09.064617 11871 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:09.064674 11871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:09.064826 11871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:09.066691 11871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "da554ea29e0a487c9805c1e8d0863b2b"
format_stamp: "Formatted at 2026-08-06 12:29:09 on dist-test-slave-1xrh"
server_key: "fd5c4eebc173404bbc5d5cb5e00ece39"
server_key_iv: "f3c1e2dc755abd8565ad514cf89bc60d"
server_key_version: "encryptionkey@0"
I20260806 12:29:09.067272 11871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "da554ea29e0a487c9805c1e8d0863b2b"
format_stamp: "Formatted at 2026-08-06 12:29:09 on dist-test-slave-1xrh"
server_key: "fd5c4eebc173404bbc5d5cb5e00ece39"
server_key_iv: "f3c1e2dc755abd8565ad514cf89bc60d"
server_key_version: "encryptionkey@0"
I20260806 12:29:09.070786 11871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 12:29:09.073148 11885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:09.074309 11871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:09.074397 11871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "da554ea29e0a487c9805c1e8d0863b2b"
format_stamp: "Formatted at 2026-08-06 12:29:09 on dist-test-slave-1xrh"
server_key: "fd5c4eebc173404bbc5d5cb5e00ece39"
server_key_iv: "f3c1e2dc755abd8565ad514cf89bc60d"
server_key_version: "encryptionkey@0"
I20260806 12:29:09.074488 11871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:09.078729 11871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:09.079366 11871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:09.079504 11871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:09.079957 11871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:09.080751 11871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:09.080796 11871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:09.080832 11871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:09.080852 11871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:09.089268 11871 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:34963
I20260806 12:29:09.089346 12002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:34963 every 8 connection(s)
I20260806 12:29:09.090138 11871 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:09.096062 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.091937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:57001 (local address 127.6.152.190:36429)
0806 12:29:09.092082 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.092085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.093044 (+ 959us) server_negotiation.cc:407] Connection header received
0806 12:29:09.094359 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.094363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.094411 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.094517 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.095606 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.095611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.095618 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.095629 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.095648 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.095656 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.095747 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.095879 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.095926 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:29:09.096837 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.092286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:36429 (local address 127.6.152.130:57001)
0806 12:29:09.092900 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.092937 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.094146 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.094685 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.094692 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.094772 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.095332 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.095333 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.095369 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.095431 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.095733 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.095741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.095747 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.095919 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.096150 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
I20260806 12:29:09.097854 12003 heartbeater.cc:347] Connected to a master server at 127.6.152.190:36429
I20260806 12:29:09.098079 12003 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:09.098238 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 11871
I20260806 12:29:09.098366 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:09.098510 12003 heartbeater.cc:511] Master 127.6.152.190:36429 requested a full tablet report, sending...
I20260806 12:29:09.098670 6754 external_mini_cluster.cc:1428] Setting key d77664c1eb596a619677769fca24e413
I20260806 12:29:09.099527 11669 ts_manager.cc:194] Registered new tserver with Master: da554ea29e0a487c9805c1e8d0863b2b (127.6.152.130:34963)
I20260806 12:29:09.100477 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36429
--builtin_ntp_servers=127.6.152.148:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 12:29:09.210282 12007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:09.210673 12007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:09.210757 12007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:09.214712 12007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:09.214789 12007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:09.214884 12007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:09.219810 12007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:09.220968 12007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:09.222330 12007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:09.231662 12013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:09.231715 12014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:09.231678 12018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:09.231861 12007 server_base.cc:1034] running on GCE node
I20260806 12:29:09.232296 12007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:09.232926 12007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:09.234104 12007 hybrid_clock.cc:648] HybridClock initialized: now 1786019349234077 us; error 40 us; skew 500 ppm
I20260806 12:29:09.235832 12007 webserver.cc:460] Webserver started at http://127.6.152.131:42901/ using document root <none> and password file <none>
I20260806 12:29:09.236397 12007 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:09.236487 12007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:09.236712 12007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:09.238636 12007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "1b98df8594e74b32abd057ff938a4597"
format_stamp: "Formatted at 2026-08-06 12:29:09 on dist-test-slave-1xrh"
server_key: "30b8bbdcc228572724bd5595d7fba2ce"
server_key_iv: "d6a7b7db08b0b0228152daec62f298b1"
server_key_version: "encryptionkey@0"
I20260806 12:29:09.239298 12007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "1b98df8594e74b32abd057ff938a4597"
format_stamp: "Formatted at 2026-08-06 12:29:09 on dist-test-slave-1xrh"
server_key: "30b8bbdcc228572724bd5595d7fba2ce"
server_key_iv: "d6a7b7db08b0b0228152daec62f298b1"
server_key_version: "encryptionkey@0"
I20260806 12:29:09.242749 12007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:29:09.245100 12024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:09.246243 12007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:09.246376 12007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "1b98df8594e74b32abd057ff938a4597"
format_stamp: "Formatted at 2026-08-06 12:29:09 on dist-test-slave-1xrh"
server_key: "30b8bbdcc228572724bd5595d7fba2ce"
server_key_iv: "d6a7b7db08b0b0228152daec62f298b1"
server_key_version: "encryptionkey@0"
I20260806 12:29:09.246482 12007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:09.250849 12007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:09.251504 12007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:09.251709 12007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:09.252290 12007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:09.253180 12007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:09.253226 12007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:09.253288 12007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:09.253322 12007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:09.262310 12007 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:44181
I20260806 12:29:09.262332 12141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:44181 every 8 connection(s)
I20260806 12:29:09.263226 12007 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:09.266665 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12007
I20260806 12:29:09.266799 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:09.267071 6754 external_mini_cluster.cc:1428] Setting key 1a9291f6e8027d0d0e977fbffdd188e4
I20260806 12:29:09.269333 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.265119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:41445 (local address 127.6.152.190:36429)
0806 12:29:09.265261 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.265265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.266217 (+ 952us) server_negotiation.cc:407] Connection header received
0806 12:29:09.267668 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.267672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.267715 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.267796 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.268894 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.268896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.268898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.268919 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.268937 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.268944 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.269012 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.269165 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.269199 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:09.270114 12144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.265471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:36429 (local address 127.6.152.131:41445)
0806 12:29:09.266075 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.266111 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.267437 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.267985 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.267993 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.268103 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.268625 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.268627 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.268663 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.268757 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.269027 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.269036 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.269041 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.269204 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.269426 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":98,"threads_started":1}
I20260806 12:29:09.271049 12142 heartbeater.cc:347] Connected to a master server at 127.6.152.190:36429
I20260806 12:29:09.271265 12142 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:09.271612 12142 heartbeater.cc:511] Master 127.6.152.190:36429 requested a full tablet report, sending...
I20260806 12:29:09.272483 11669 ts_manager.cc:194] Registered new tserver with Master: 1b98df8594e74b32abd057ff938a4597 (127.6.152.131:44181)
I20260806 12:29:09.280941 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:29:09.284423 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.282744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.6.152.190:36429)
0806 12:29:09.282890 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.282895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.283087 (+ 192us) server_negotiation.cc:407] Connection header received
0806 12:29:09.283272 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.283275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.283309 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.283395 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.284010 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.284014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.284017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.284029 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.284047 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.284056 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.284153 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.284262 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.284300 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:09.287029 11669 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:09.290134 11669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:09.301739 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.298427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44181 (local address 127.0.0.1:47162)
0806 12:29:09.299390 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.299406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.299507 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.300025 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.300031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.300101 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.300745 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.300747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.300790 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.301016 (+ 226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.301389 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.301399 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.301408 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.301522 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.301590 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":170,"threads_started":1}
I20260806 12:29:09.301761 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.299046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34459 (local address 127.0.0.1:50096)
0806 12:29:09.299716 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.299731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.299806 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.300513 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.300516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.300530 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.300885 (+ 355us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.300887 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.300894 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.300908 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.301420 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.301424 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.301427 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.301523 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.301601 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260806 12:29:09.302116 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.298956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47162 (local address 127.6.152.131:44181)
0806 12:29:09.299309 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.299324 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.299620 (+ 296us) server_negotiation.cc:407] Connection header received
0806 12:29:09.299686 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.299691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.299879 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.300029 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.301138 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.301144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.301155 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.301167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.301210 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.301246 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.301344 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.301686 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.301759 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":126,"threads_started":1}
I20260806 12:29:09.302345 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.298311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.6.152.130:34963)
0806 12:29:09.299659 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.299673 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.299698 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:29:09.299759 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.299768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.299962 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.300734 (+ 772us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.301298 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.301305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.301313 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.301323 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.301357 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.301383 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.302017 (+ 634us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.302100 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.302171 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1221,"thread_start_us":263,"threads_started":1}
I20260806 12:29:09.302361 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.298171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:34963 (local address 127.0.0.1:38298)
0806 12:29:09.298857 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.298905 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.299013 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.300129 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.300132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.300151 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.301068 (+ 917us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.301070 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.301082 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.301113 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.301505 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.301508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.301511 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.301577 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.301603 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":83,"threads_started":1}
I20260806 12:29:09.302815 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.299537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.6.152.129:34459)
0806 12:29:09.300086 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.300100 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.300123 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:29:09.300174 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.300179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.300365 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.300513 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.301228 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.301234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.301254 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.301263 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.301300 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.301320 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.301447 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.302597 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.302662 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":104,"threads_started":1}
I20260806 12:29:09.304898 12072 tablet_service.cc:1467] Processing CreateTablet for tablet c7673aef12c04dcbbd35a95f2e413844 (DEFAULT_TABLE table=test-table [id=637bcfb813524521bf40b05081ef49cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:09.305227 11935 tablet_service.cc:1467] Processing CreateTablet for tablet c7673aef12c04dcbbd35a95f2e413844 (DEFAULT_TABLE table=test-table [id=637bcfb813524521bf40b05081ef49cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:09.305337 11798 tablet_service.cc:1467] Processing CreateTablet for tablet c7673aef12c04dcbbd35a95f2e413844 (DEFAULT_TABLE table=test-table [id=637bcfb813524521bf40b05081ef49cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:09.306026 12072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7673aef12c04dcbbd35a95f2e413844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:09.306396 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7673aef12c04dcbbd35a95f2e413844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:09.306422 11798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7673aef12c04dcbbd35a95f2e413844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:09.313851 12160 tablet_bootstrap.cc:492] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: Bootstrap starting.
I20260806 12:29:09.314841 12159 tablet_bootstrap.cc:492] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Bootstrap starting.
I20260806 12:29:09.315349 12161 tablet_bootstrap.cc:492] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Bootstrap starting.
I20260806 12:29:09.317019 12159 tablet_bootstrap.cc:654] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:09.317019 12160 tablet_bootstrap.cc:654] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:09.317950 12159 log.cc:824] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:09.317950 12160 log.cc:824] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:09.318738 12161 tablet_bootstrap.cc:654] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:09.319591 12161 log.cc:824] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:09.320676 12160 tablet_bootstrap.cc:492] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: No bootstrap required, opened a new log
I20260806 12:29:09.320930 12159 tablet_bootstrap.cc:492] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: No bootstrap required, opened a new log
I20260806 12:29:09.320930 12160 ts_tablet_manager.cc:1397] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 12:29:09.321089 12159 ts_tablet_manager.cc:1397] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:29:09.321476 12161 tablet_bootstrap.cc:492] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: No bootstrap required, opened a new log
I20260806 12:29:09.321667 12161 ts_tablet_manager.cc:1397] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 12:29:09.325047 12161 raft_consensus.cc:348] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.325280 12161 raft_consensus.cc:374] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:09.325335 12161 raft_consensus.cc:729] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b98df8594e74b32abd057ff938a4597, State: Initialized, Role: FOLLOWER
I20260806 12:29:09.325547 12160 raft_consensus.cc:348] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.325547 12159 raft_consensus.cc:348] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.325794 12159 raft_consensus.cc:374] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:09.325800 12160 raft_consensus.cc:374] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:09.325855 12159 raft_consensus.cc:729] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da554ea29e0a487c9805c1e8d0863b2b, State: Initialized, Role: FOLLOWER
I20260806 12:29:09.325855 12160 raft_consensus.cc:729] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 497d9a0be56d4abd93f6649fb70121c3, State: Initialized, Role: FOLLOWER
I20260806 12:29:09.325805 12161 consensus_queue.cc:260] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.326267 12160 consensus_queue.cc:260] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.326267 12159 consensus_queue.cc:260] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.326941 12161 ts_tablet_manager.cc:1428] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:29:09.327176 12159 ts_tablet_manager.cc:1428] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 12:29:09.327191 12160 ts_tablet_manager.cc:1428] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:29:09.327296 12142 heartbeater.cc:503] Master 127.6.152.190:36429 was elected leader, sending a full tablet report...
I20260806 12:29:09.327471 12003 heartbeater.cc:503] Master 127.6.152.190:36429 was elected leader, sending a full tablet report...
I20260806 12:29:09.328317 11868 heartbeater.cc:503] Master 127.6.152.190:36429 was elected leader, sending a full tablet report...
W20260806 12:29:09.341015 12004 tablet.cc:2369] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:29:09.423924 11869 tablet.cc:2369] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:29:09.514277 12143 tablet.cc:2369] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:29:09.733748 12168 raft_consensus.cc:482] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:09.733955 12168 raft_consensus.cc:504] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.735198 12168 leader_election.cc:288] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 497d9a0be56d4abd93f6649fb70121c3 (127.6.152.129:34459), 1b98df8594e74b32abd057ff938a4597 (127.6.152.131:44181)
I20260806 12:29:09.737581 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.735661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:38265 (local address 127.6.152.129:34459)
0806 12:29:09.735821 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.735826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.735975 (+ 149us) server_negotiation.cc:407] Connection header received
0806 12:29:09.736150 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.736154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.736197 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.736288 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.737107 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.737111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.737114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.737128 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.737150 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.737160 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.737253 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.737360 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.737432 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:29:09.737576 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.735503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34459 (local address 127.6.152.130:38265)
0806 12:29:09.735871 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.735893 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.736024 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.736310 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.736313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.736328 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.736893 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.736896 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.736910 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.736947 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.737262 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.737265 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.737269 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.737355 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.737400 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":82,"threads_started":1}
I20260806 12:29:09.737690 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.735753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:54501 (local address 127.6.152.131:44181)
0806 12:29:09.735933 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.735939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.736119 (+ 180us) server_negotiation.cc:407] Connection header received
0806 12:29:09.736252 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.736256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.736298 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.736385 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.737276 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.737280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.737283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.737296 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.737320 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.737334 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.737418 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.737507 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.737543 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:29:09.737727 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.735522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44181 (local address 127.6.152.130:54501)
0806 12:29:09.736023 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.736035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:09.736143 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:09.736411 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:09.736414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:09.736430 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:09.737104 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:09.737106 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:09.737117 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:09.737143 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:09.737424 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:09.737428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:09.737431 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 12:29:09.737512 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 12:29:09.737546 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":76,"threads_started":1}
I20260806 12:29:09.738511 11819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7673aef12c04dcbbd35a95f2e413844" candidate_uuid: "da554ea29e0a487c9805c1e8d0863b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "497d9a0be56d4abd93f6649fb70121c3" is_pre_election: true
I20260806 12:29:09.738511 12094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7673aef12c04dcbbd35a95f2e413844" candidate_uuid: "da554ea29e0a487c9805c1e8d0863b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b98df8594e74b32abd057ff938a4597" is_pre_election: true
I20260806 12:29:09.738816 11819 raft_consensus.cc:2454] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da554ea29e0a487c9805c1e8d0863b2b in term 0.
I20260806 12:29:09.738816 12094 raft_consensus.cc:2454] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da554ea29e0a487c9805c1e8d0863b2b in term 0.
I20260806 12:29:09.739353 11889 leader_election.cc:302] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b98df8594e74b32abd057ff938a4597, 497d9a0be56d4abd93f6649fb70121c3, da554ea29e0a487c9805c1e8d0863b2b; no voters:
I20260806 12:29:09.739662 12168 raft_consensus.cc:2781] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:09.739770 12168 raft_consensus.cc:482] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:09.739832 12168 raft_consensus.cc:3037] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:09.740938 12168 raft_consensus.cc:504] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.741331 12168 leader_election.cc:288] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [CANDIDATE]: Term 1 election: Requested vote from peers 497d9a0be56d4abd93f6649fb70121c3 (127.6.152.129:34459), 1b98df8594e74b32abd057ff938a4597 (127.6.152.131:44181)
I20260806 12:29:09.741714 12094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7673aef12c04dcbbd35a95f2e413844" candidate_uuid: "da554ea29e0a487c9805c1e8d0863b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b98df8594e74b32abd057ff938a4597"
I20260806 12:29:09.741851 12094 raft_consensus.cc:3037] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:09.742894 12094 raft_consensus.cc:2454] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da554ea29e0a487c9805c1e8d0863b2b in term 1.
I20260806 12:29:09.743211 11819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7673aef12c04dcbbd35a95f2e413844" candidate_uuid: "da554ea29e0a487c9805c1e8d0863b2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "497d9a0be56d4abd93f6649fb70121c3"
I20260806 12:29:09.743263 11889 leader_election.cc:302] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b98df8594e74b32abd057ff938a4597, da554ea29e0a487c9805c1e8d0863b2b; no voters:
I20260806 12:29:09.743373 11819 raft_consensus.cc:3037] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:09.743462 12168 raft_consensus.cc:2781] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:09.743722 12168 raft_consensus.cc:686] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 1 LEADER]: Becoming Leader. State: Replica: da554ea29e0a487c9805c1e8d0863b2b, State: Running, Role: LEADER
I20260806 12:29:09.744096 12168 consensus_queue.cc:237] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } }
I20260806 12:29:09.744800 11819 raft_consensus.cc:2454] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da554ea29e0a487c9805c1e8d0863b2b in term 1.
I20260806 12:29:09.747658 11669 catalog_manager.cc:5495] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b reported cstate change: term changed from 0 to 1, leader changed from <none> to da554ea29e0a487c9805c1e8d0863b2b (127.6.152.130). New cstate: current_term: 1 leader_uuid: "da554ea29e0a487c9805c1e8d0863b2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da554ea29e0a487c9805c1e8d0863b2b" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 34963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:09.861514 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.859864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.6.152.130:34963)
0806 12:29:09.860115 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:09.860119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:09.860134 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:29:09.860296 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:09.860299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:09.860360 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:09.860444 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:09.861112 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:09.861115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:09.861117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:09.861128 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:09.861148 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:09.861156 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:09.861230 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:09.861287 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 12:29:09.861320 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":65,"threads_started":1}
I20260806 12:29:09.867265 12094 raft_consensus.cc:1261] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 1 FOLLOWER]: Refusing update from remote peer da554ea29e0a487c9805c1e8d0863b2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:09.867266 11819 raft_consensus.cc:1261] T c7673aef12c04dcbbd35a95f2e413844 P 497d9a0be56d4abd93f6649fb70121c3 [term 1 FOLLOWER]: Refusing update from remote peer da554ea29e0a487c9805c1e8d0863b2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:09.867971 12172 consensus_queue.cc:1035] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "497d9a0be56d4abd93f6649fb70121c3" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:09.868177 12168 consensus_queue.cc:1035] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b98df8594e74b32abd057ff938a4597" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:09.874548 12177 mvcc.cc:205] Tried to move back new op lower bound from 7315535257049120768 to 7315535256555376640. Current Snapshot: MvccSnapshot[applied={T|T < 7315535257049120768}]
I20260806 12:29:09.876844 12180 mvcc.cc:205] Tried to move back new op lower bound from 7315535257049120768 to 7315535256555376640. Current Snapshot: MvccSnapshot[applied={T|T < 7315535257049120768}]
I20260806 12:29:09.884511 11670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:09.884654 11670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:09.887729 11670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33f743a4705941aca8ac40cdbf774cbb: Sending DeleteTablet for 3 replicas of tablet c7673aef12c04dcbbd35a95f2e413844
I20260806 12:29:09.888569 11798 tablet_service.cc:1514] Processing DeleteTablet for tablet c7673aef12c04dcbbd35a95f2e413844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:09 UTC) from {username='slave'} at 127.0.0.1:50096
I20260806 12:29:09.888689 12072 tablet_service.cc:1514] Processing DeleteTablet for tablet c7673aef12c04dcbbd35a95f2e413844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:09 UTC) from {username='slave'} at 127.0.0.1:47162
I20260806 12:29:09.888687 11935 tablet_service.cc:1514] Processing DeleteTablet for tablet c7673aef12c04dcbbd35a95f2e413844 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:09 UTC) from {username='slave'} at 127.0.0.1:38298
I20260806 12:29:09.889213 12192 tablet_replica.cc:330] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: stopping tablet replica
I20260806 12:29:09.889500 12192 raft_consensus.cc:2229] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:09.889868 12192 raft_consensus.cc:2258] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:09.889977 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11733
I20260806 12:29:09.894421 12192 ts_tablet_manager.cc:1905] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:09.894757 12191 tablet_replica.cc:330] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: stopping tablet replica
I20260806 12:29:09.895114 12191 raft_consensus.cc:2229] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:09.895530 12191 raft_consensus.cc:2258] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:09.896844 12191 ts_tablet_manager.cc:1905] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:09.897351 12192 ts_tablet_manager.cc:1918] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:09.897467 12192 log.cc:1196] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/c7673aef12c04dcbbd35a95f2e413844
I20260806 12:29:09.897784 12192 ts_tablet_manager.cc:1939] T c7673aef12c04dcbbd35a95f2e413844 P da554ea29e0a487c9805c1e8d0863b2b: Deleting consensus metadata
I20260806 12:29:09.898965 11653 catalog_manager.cc:4841] TS da554ea29e0a487c9805c1e8d0863b2b (127.6.152.130:34963): tablet c7673aef12c04dcbbd35a95f2e413844 (table test-table [id=637bcfb813524521bf40b05081ef49cc]) successfully deleted
I20260806 12:29:09.899752 12191 ts_tablet_manager.cc:1918] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:09.899875 12191 log.cc:1196] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/c7673aef12c04dcbbd35a95f2e413844
I20260806 12:29:09.900159 12191 ts_tablet_manager.cc:1939] T c7673aef12c04dcbbd35a95f2e413844 P 1b98df8594e74b32abd057ff938a4597: Deleting consensus metadata
W20260806 12:29:09.900982 11653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:34459 (error 108)
I20260806 12:29:09.901352 11655 catalog_manager.cc:4841] TS 1b98df8594e74b32abd057ff938a4597 (127.6.152.131:44181): tablet c7673aef12c04dcbbd35a95f2e413844 (table test-table [id=637bcfb813524521bf40b05081ef49cc]) successfully deleted
I20260806 12:29:09.901753 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11871
I20260806 12:29:09.905784 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:09.901783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34459 (local address 127.0.0.1:49460)
0806 12:29:09.905635 (+ 3852us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:09.905703 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3781,"thread_start_us":56,"threads_started":1}
W20260806 12:29:09.905949 11653 catalog_manager.cc:4584] TS 497d9a0be56d4abd93f6649fb70121c3 (127.6.152.129:34459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7673aef12c04dcbbd35a95f2e413844: Network error: Client connection negotiation failed: client connection to 127.6.152.129:34459: connect: Connection refused (error 111)
I20260806 12:29:09.911522 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12007
I20260806 12:29:09.918026 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 11637
2026-08-06T12:29:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1410 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 12:29:09.940150 6754 mini_oidc.cc:152] Starting JWKS server
I20260806 12:29:09.941923 6754 webserver.cc:460] Webserver started at https://127.0.0.1:44931/ using document root <none> and password file <none>
I20260806 12:29:09.942118 6754 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 12:29:09.942399 6754 webserver.cc:460] Webserver started at http://127.0.0.1:34329/ using document root <none> and password file <none>
2026-08-06T12:29:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:09Z Disabled control of system clock
I20260806 12:29:09.951689 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39141
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:34289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:39141
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44931/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/testchainca.cert with env {}
W20260806 12:29:10.062693 12201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:10.063022 12201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:10.063093 12201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:10.067166 12201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:10.067255 12201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:10.067286 12201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:10.067327 12201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:10.067377 12201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:29:10.067414 12201 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:29:10.067446 12201 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44931/jwks.json
I20260806 12:29:10.072865 12201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:39141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44931/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:10.073968 12201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:10.075045 12201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:10.083622 12207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:10.083650 12206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:10.083709 12201 server_base.cc:1034] running on GCE node
W20260806 12:29:10.083650 12209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:10.084478 12201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:10.085568 12201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:10.086736 12201 hybrid_clock.cc:648] HybridClock initialized: now 1786019350086718 us; error 31 us; skew 500 ppm
I20260806 12:29:10.088611 12201 webserver.cc:460] Webserver started at http://127.6.152.190:44509/ using document root <none> and password file <none>
I20260806 12:29:10.089176 12201 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:10.089241 12201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:10.089473 12201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:10.091508 12201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "1ef045471035461993b7d8a39239a2c0"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "42fc4f401048c2206d7c7b6f1d88a463"
server_key_iv: "8ddeceeba46ce3f4c35df078d124fb60"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.092092 12201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "1ef045471035461993b7d8a39239a2c0"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "42fc4f401048c2206d7c7b6f1d88a463"
server_key_iv: "8ddeceeba46ce3f4c35df078d124fb60"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.095641 12201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:29:10.097892 12215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:10.099007 12201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:10.099172 12201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "1ef045471035461993b7d8a39239a2c0"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "42fc4f401048c2206d7c7b6f1d88a463"
server_key_iv: "8ddeceeba46ce3f4c35df078d124fb60"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.099293 12201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:10.110297 12201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:10.110900 12201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:10.111092 12201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:10.118233 12201 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:39141
I20260806 12:29:10.118237 12273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:39141 every 8 connection(s)
I20260806 12:29:10.119146 12201 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:10.121915 12274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:10.127058 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12201
I20260806 12:29:10.127213 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:10.127506 6754 external_mini_cluster.cc:1428] Setting key 68d6656a3a62e80a4756514537a28e49
I20260806 12:29:10.129061 12274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0: Bootstrap starting.
I20260806 12:29:10.131480 12274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:10.132241 12274 log.cc:824] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:10.134436 12274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0: No bootstrap required, opened a new log
I20260806 12:29:10.137913 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:10.128990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.6.152.190:39141)
0806 12:29:10.129704 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:10.129720 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:10.129757 (+ 37us) server_negotiation.cc:407] Connection header received
0806 12:29:10.130410 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:10.130426 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:10.130822 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:10.131214 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:10.131710 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.132815 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.134097 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.134615 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.134737 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.135658 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:10.135663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:10.135676 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:10.136071 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:10.136127 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:10.136147 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:10.136226 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:10.136398 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 12:29:10.136674 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":150,"threads_started":1}
I20260806 12:29:10.137820 12274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef045471035461993b7d8a39239a2c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39141 } }
I20260806 12:29:10.138051 12274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:10.138109 12274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef045471035461993b7d8a39239a2c0, State: Initialized, Role: FOLLOWER
I20260806 12:29:10.138564 12274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef045471035461993b7d8a39239a2c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39141 } }
I20260806 12:29:10.138685 12274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:10.138769 12274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:10.138865 12274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:10.139989 12274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef045471035461993b7d8a39239a2c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39141 } }
I20260806 12:29:10.140343 12274 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef045471035461993b7d8a39239a2c0; no voters:
I20260806 12:29:10.140687 12274 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:10.140810 12280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:10.140954 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:29:10.141057 12280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef045471035461993b7d8a39239a2c0, State: Running, Role: LEADER
I20260806 12:29:10.141337 12280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef045471035461993b7d8a39239a2c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39141 } }
I20260806 12:29:10.141990 12274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:10.142684 12282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef045471035461993b7d8a39239a2c0. Latest consensus state: current_term: 1 leader_uuid: "1ef045471035461993b7d8a39239a2c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef045471035461993b7d8a39239a2c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39141 } } }
I20260806 12:29:10.142915 12282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:10.142939 12281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef045471035461993b7d8a39239a2c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef045471035461993b7d8a39239a2c0" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39141 } } }
I20260806 12:29:10.143069 12281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:10.143383 12289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:10.146595 12289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:10.152163 12289 catalog_manager.cc:1347] Generated new cluster ID: a9b99c5a5f4242de9314c55dfbc9034e
I20260806 12:29:10.152235 12289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:10.162134 12289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:10.162997 12289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:10.169473 12289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef045471035461993b7d8a39239a2c0: Generated new TSK 0
I20260806 12:29:10.170094 12289 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 12:29:10.201253 12307 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 12:29:10.201414 12307 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 12:29:10.202062 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:10.183701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.6.152.190:39141)
0806 12:29:10.183842 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:10.183847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:10.184022 (+ 175us) server_negotiation.cc:407] Connection header received
0806 12:29:10.184203 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:10.184207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:10.184260 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:10.184367 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:29:10.184594 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.185403 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.186276 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.186644 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.186750 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.187050 (+ 300us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:29:10.201646 (+ 14596us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 12:29:10.201921 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 12:29:10.202154 12277 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 12:29:10.203517 12306 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:10.183613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:39141 (local address 127.0.0.1:52734)
0806 12:29:10.183940 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:10.183954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:10.184061 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:10.184385 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:10.184387 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:10.184393 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:29:10.184486 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:10.184491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.185524 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.185527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:10.186148 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:10.186155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.186761 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.186764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:10.186828 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.186871 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:29:10.201890 (+ 15019us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 12:29:10.202096 (+ 206us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.152.190:39141: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":112,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.152.190:39141: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44931/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 12:29:10.204618 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12201
2026-08-06T12:29:10Z chronyd exiting
I20260806 12:29:10.242949 6754 test_util.cc:175] -----------------------------------------------
I20260806 12:29:10.243031 6754 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786019324359312-6754-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (304 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 12:29:10.244203 6754 mini_oidc.cc:152] Starting JWKS server
I20260806 12:29:10.245064 6754 webserver.cc:460] Webserver started at https://127.0.0.1:33795/ using document root <none> and password file <none>
I20260806 12:29:10.245190 6754 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 12:29:10.245453 6754 webserver.cc:460] Webserver started at http://127.0.0.1:41665/ using document root <none> and password file <none>
2026-08-06T12:29:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:10Z Disabled control of system clock
I20260806 12:29:10.254782 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37937
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:34501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37937
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33795/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/testchainca.cert with env {}
W20260806 12:29:10.365885 12315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:10.366142 12315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:10.366182 12315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:10.370059 12315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:10.370110 12315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:10.370126 12315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:10.370142 12315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:10.370157 12315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:29:10.370173 12315 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:29:10.370187 12315 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33795/jwks.json
I20260806 12:29:10.375560 12315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33795/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:10.376565 12315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:10.377521 12315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:10.386462 12320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:10.386428 12323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:10.386428 12321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:10.386468 12315 server_base.cc:1034] running on GCE node
I20260806 12:29:10.387305 12315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:10.388330 12315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:10.389492 12315 hybrid_clock.cc:648] HybridClock initialized: now 1786019350389472 us; error 32 us; skew 500 ppm
I20260806 12:29:10.391287 12315 webserver.cc:460] Webserver started at http://127.6.152.190:42879/ using document root <none> and password file <none>
I20260806 12:29:10.391891 12315 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:10.391956 12315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:10.392197 12315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:10.394142 12315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "0d821865e15b4bb1b375f9191373b96f"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "71a4069dfc96dc7c171e911b93f75a41"
server_key_iv: "7d10d093fcd15ea8129abd122dcbbcb7"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.394692 12315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "0d821865e15b4bb1b375f9191373b96f"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "71a4069dfc96dc7c171e911b93f75a41"
server_key_iv: "7d10d093fcd15ea8129abd122dcbbcb7"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.398272 12315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:29:10.400787 12329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:10.401943 12315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:10.402083 12315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "0d821865e15b4bb1b375f9191373b96f"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "71a4069dfc96dc7c171e911b93f75a41"
server_key_iv: "7d10d093fcd15ea8129abd122dcbbcb7"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.402215 12315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:10.422001 12315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:10.422741 12315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:10.422942 12315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:10.429943 12315 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37937
I20260806 12:29:10.429961 12383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37937 every 8 connection(s)
I20260806 12:29:10.431023 12315 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:10.434079 12384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:10.439502 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f: Bootstrap starting.
I20260806 12:29:10.440579 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12315
I20260806 12:29:10.440678 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:10.440990 6754 external_mini_cluster.cc:1428] Setting key 5b8e2cb7d6bcf6563d34bb31b9dd706b
I20260806 12:29:10.442179 12384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:10.442927 12384 log.cc:824] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:10.445065 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f: No bootstrap required, opened a new log
I20260806 12:29:10.449378 12384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d821865e15b4bb1b375f9191373b96f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37937 } }
I20260806 12:29:10.449687 12384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:10.449762 12384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d821865e15b4bb1b375f9191373b96f, State: Initialized, Role: FOLLOWER
I20260806 12:29:10.450197 12384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d821865e15b4bb1b375f9191373b96f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37937 } }
I20260806 12:29:10.450315 12384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:10.450393 12384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:10.450511 12384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:10.451050 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:10.442368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.6.152.190:37937)
0806 12:29:10.443023 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:10.443061 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:10.443113 (+ 52us) server_negotiation.cc:407] Connection header received
0806 12:29:10.443866 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:10.443886 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:10.444210 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:10.444488 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:10.444824 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.445936 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.447251 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.447742 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.447940 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.448903 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:10.448908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:10.448921 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:10.449299 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:10.449354 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:10.449374 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:10.449451 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:10.449626 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 12:29:10.450147 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":153,"threads_started":1}
I20260806 12:29:10.451488 12384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d821865e15b4bb1b375f9191373b96f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37937 } }
I20260806 12:29:10.451809 12384 leader_election.cc:302] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d821865e15b4bb1b375f9191373b96f; no voters:
I20260806 12:29:10.452101 12384 leader_election.cc:288] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:10.452236 12389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:10.452463 12389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [term 1 LEADER]: Becoming Leader. State: Replica: 0d821865e15b4bb1b375f9191373b96f, State: Running, Role: LEADER
I20260806 12:29:10.452749 12389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d821865e15b4bb1b375f9191373b96f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37937 } }
I20260806 12:29:10.453436 12384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:10.454101 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:29:10.454209 12389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d821865e15b4bb1b375f9191373b96f. Latest consensus state: current_term: 1 leader_uuid: "0d821865e15b4bb1b375f9191373b96f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d821865e15b4bb1b375f9191373b96f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37937 } } }
I20260806 12:29:10.454321 12389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:10.454756 12390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d821865e15b4bb1b375f9191373b96f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d821865e15b4bb1b375f9191373b96f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37937 } } }
I20260806 12:29:10.454900 12390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:10.455364 12396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:10.458491 12396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:10.464119 12396 catalog_manager.cc:1347] Generated new cluster ID: aca9cbb57b9940a4abebae9640a59d6b
I20260806 12:29:10.464193 12396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:10.473664 12396 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:10.474650 12396 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:10.478897 12396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d821865e15b4bb1b375f9191373b96f: Generated new TSK 0
I20260806 12:29:10.479602 12396 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 12:29:10.514951 12414 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 12:29:10.515051 12414 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 12:29:10.515769 12387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:10.494850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.6.152.190:37937)
0806 12:29:10.494985 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:10.494990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:10.495198 (+ 208us) server_negotiation.cc:407] Connection header received
0806 12:29:10.495353 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:10.495355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:10.495402 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:10.495485 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:29:10.495742 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.496280 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.497496 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.497719 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.497799 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.498125 (+ 326us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:29:10.515361 (+ 17236us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 12:29:10.515632 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52344: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 12:29:10.515836 12387 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52344: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 12:29:10.515811 12413 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:10.494770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37937 (local address 127.0.0.1:52344)
0806 12:29:10.495093 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:10.495108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:10.495248 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:10.495513 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:10.495516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:10.495524 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:29:10.495637 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:10.495643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.496394 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.496398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:10.497363 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:10.497374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.497835 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.497839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:10.497938 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.497967 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:29:10.515542 (+ 17575us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 12:29:10.515679 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.152.190:37937: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33795/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":124,"threads_started":1}
I20260806 12:29:10.516286 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12315
2026-08-06T12:29:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (312 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 12:29:10.556589 6754 mini_oidc.cc:152] Starting JWKS server
I20260806 12:29:10.557482 6754 webserver.cc:460] Webserver started at https://127.0.0.1:39063/ using document root <none> and password file <none>
I20260806 12:29:10.557631 6754 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 12:29:10.557902 6754 webserver.cc:460] Webserver started at http://127.0.0.1:44927/ using document root <none> and password file <none>
2026-08-06T12:29:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:10Z Disabled control of system clock
I20260806 12:29:10.567528 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:40273
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:46463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:40273
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39063/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 12:29:10.677738 12421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:10.678004 12421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:10.678042 12421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:10.682082 12421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:10.682135 12421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:10.682152 12421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:10.682168 12421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:10.682183 12421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:29:10.682199 12421 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:29:10.682250 12421 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39063/jwks.json
I20260806 12:29:10.686990 12421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:40273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:40273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39063/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:10.687995 12421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:10.689267 12421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:10.697602 12429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:10.697789 12421 server_base.cc:1034] running on GCE node
W20260806 12:29:10.697676 12427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:10.697608 12426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:10.698477 12421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:10.699551 12421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:10.700749 12421 hybrid_clock.cc:648] HybridClock initialized: now 1786019350700708 us; error 52 us; skew 500 ppm
I20260806 12:29:10.702510 12421 webserver.cc:460] Webserver started at http://127.6.152.190:35025/ using document root <none> and password file <none>
I20260806 12:29:10.703063 12421 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:10.703182 12421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:10.703423 12421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:10.705370 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "e2c64b24b14349c5b9b5e936888737c4"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "ed22f9377ea6454aaf2771eedbb94168"
server_key_iv: "10c28561a134b531d2c0f741d6b4d9f0"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.705961 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "e2c64b24b14349c5b9b5e936888737c4"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "ed22f9377ea6454aaf2771eedbb94168"
server_key_iv: "10c28561a134b531d2c0f741d6b4d9f0"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.709662 12421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:29:10.711995 12436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:10.713276 12421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:10.713428 12421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "e2c64b24b14349c5b9b5e936888737c4"
format_stamp: "Formatted at 2026-08-06 12:29:10 on dist-test-slave-1xrh"
server_key: "ed22f9377ea6454aaf2771eedbb94168"
server_key_iv: "10c28561a134b531d2c0f741d6b4d9f0"
server_key_version: "encryptionkey@0"
I20260806 12:29:10.713538 12421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:10.741112 12421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:10.741839 12421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:10.742046 12421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:10.749344 12421 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:40273
I20260806 12:29:10.749366 12490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:40273 every 8 connection(s)
I20260806 12:29:10.750267 12421 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:10.753099 12491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:10.753499 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12421
I20260806 12:29:10.753648 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:10.753966 6754 external_mini_cluster.cc:1428] Setting key c708d31d548c6f60850d5bc4f1936b42
I20260806 12:29:10.759624 12491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4: Bootstrap starting.
I20260806 12:29:10.761970 12491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:10.762778 12491 log.cc:824] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:10.763554 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:10.755407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.6.152.190:40273)
0806 12:29:10.756332 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:10.756348 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:10.756385 (+ 37us) server_negotiation.cc:407] Connection header received
0806 12:29:10.757034 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:10.757051 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:10.757428 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:10.757756 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:10.758192 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.759230 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.760175 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.760679 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.760855 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.761726 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:10.761732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:10.761745 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:10.762162 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:10.762226 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:10.762247 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:10.762354 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:10.762502 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 12:29:10.762755 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":127,"threads_started":1}
I20260806 12:29:10.764876 12491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4: No bootstrap required, opened a new log
I20260806 12:29:10.766269 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:29:10.769207 12491 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c64b24b14349c5b9b5e936888737c4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40273 } }
I20260806 12:29:10.769527 12491 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:10.769625 12491 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c64b24b14349c5b9b5e936888737c4, State: Initialized, Role: FOLLOWER
I20260806 12:29:10.770089 12491 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c64b24b14349c5b9b5e936888737c4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40273 } }
I20260806 12:29:10.770246 12491 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:10.770349 12491 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:10.770463 12491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:10.771567 12491 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c64b24b14349c5b9b5e936888737c4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40273 } }
I20260806 12:29:10.771863 12491 leader_election.cc:302] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2c64b24b14349c5b9b5e936888737c4; no voters:
I20260806 12:29:10.772133 12491 leader_election.cc:288] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:10.772262 12496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:10.772508 12496 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [term 1 LEADER]: Becoming Leader. State: Replica: e2c64b24b14349c5b9b5e936888737c4, State: Running, Role: LEADER
I20260806 12:29:10.772804 12496 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c64b24b14349c5b9b5e936888737c4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40273 } }
I20260806 12:29:10.773566 12491 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:10.774852 12498 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c64b24b14349c5b9b5e936888737c4. Latest consensus state: current_term: 1 leader_uuid: "e2c64b24b14349c5b9b5e936888737c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c64b24b14349c5b9b5e936888737c4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40273 } } }
I20260806 12:29:10.775000 12498 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:10.775353 12497 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2c64b24b14349c5b9b5e936888737c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c64b24b14349c5b9b5e936888737c4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40273 } } }
I20260806 12:29:10.776150 12497 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:10.775450 12505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:10.778985 12505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:10.784452 12505 catalog_manager.cc:1347] Generated new cluster ID: b4cda525f14b4140821e655e23189467
I20260806 12:29:10.784514 12505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:10.794327 12505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:10.795634 12505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:10.803637 12505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2c64b24b14349c5b9b5e936888737c4: Generated new TSK 0
I20260806 12:29:10.804440 12505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:10.825565 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:10.820449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.6.152.190:40273)
0806 12:29:10.820594 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:10.820599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:10.820800 (+ 201us) server_negotiation.cc:407] Connection header received
0806 12:29:10.820979 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:10.820981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:10.821029 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:10.821118 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:29:10.821391 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.821933 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.823236 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.823456 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.823536 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.823872 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:29:10.825196 (+ 1324us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39063/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 12:29:10.825425 (+ 229us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39063/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 12:29:10.825632 12494 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39063/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 12:29:10.827428 12521 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:10.820378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40273 (local address 127.0.0.1:57888)
0806 12:29:10.820703 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:10.820717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:10.820835 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:10.821143 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:10.821146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:10.821154 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:29:10.821275 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:10.821281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.822042 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.822046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:10.823066 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:10.823076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:10.823573 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:10.823577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:10.823670 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:10.823699 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:29:10.825366 (+ 1667us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39063/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 12:29:10.827222 (+ 1856us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.152.190:40273: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39063/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1628,"thread_start_us":120,"threads_started":1}
I20260806 12:29:10.827927 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12421
E20260806 12:29:10.837957 12522 webserver.cc:533] Webserver: sslize failed:
E20260806 12:29:10.838019 12522 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T12:29:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (312 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T12:29:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:10Z Disabled control of system clock
I20260806 12:29:10.878000 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39845
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:46221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:39845
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/testchainca.cert with env {}
W20260806 12:29:10.985999 12526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:10.986235 12526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:10.986274 12526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:10.990191 12526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:10.990243 12526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:10.990258 12526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:10.990273 12526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:10.990288 12526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 12:29:10.990303 12526 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 12:29:10.990315 12526 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 12:29:10.995519 12526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:39845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:10.996492 12526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:10.997411 12526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:11.005991 12534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:11.006013 12532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:11.006026 12531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:11.006395 12526 server_base.cc:1034] running on GCE node
I20260806 12:29:11.007005 12526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:11.008134 12526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:11.009331 12526 hybrid_clock.cc:648] HybridClock initialized: now 1786019351009311 us; error 30 us; skew 500 ppm
I20260806 12:29:11.011338 12526 webserver.cc:460] Webserver started at http://127.6.152.190:38513/ using document root <none> and password file <none>
I20260806 12:29:11.011924 12526 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:11.012022 12526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:11.012261 12526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:11.014377 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "7e59153daba2445898194eb8d270932f"
format_stamp: "Formatted at 2026-08-06 12:29:11 on dist-test-slave-1xrh"
server_key: "bdbb5db56c8c8dc932058cff8b40388b"
server_key_iv: "09079be2b93d78921c496af7946cc251"
server_key_version: "encryptionkey@0"
I20260806 12:29:11.014967 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "7e59153daba2445898194eb8d270932f"
format_stamp: "Formatted at 2026-08-06 12:29:11 on dist-test-slave-1xrh"
server_key: "bdbb5db56c8c8dc932058cff8b40388b"
server_key_iv: "09079be2b93d78921c496af7946cc251"
server_key_version: "encryptionkey@0"
I20260806 12:29:11.018729 12526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:29:11.021068 12540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:11.022192 12526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 12:29:11.022341 12526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "7e59153daba2445898194eb8d270932f"
format_stamp: "Formatted at 2026-08-06 12:29:11 on dist-test-slave-1xrh"
server_key: "bdbb5db56c8c8dc932058cff8b40388b"
server_key_iv: "09079be2b93d78921c496af7946cc251"
server_key_version: "encryptionkey@0"
I20260806 12:29:11.022451 12526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:11.051611 12526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:11.052433 12526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:11.052654 12526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:11.059739 12526 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:39845
I20260806 12:29:11.059752 12597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:39845 every 8 connection(s)
I20260806 12:29:11.060674 12526 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:11.063540 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12526
I20260806 12:29:11.063721 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:11.064054 6754 external_mini_cluster.cc:1428] Setting key 9791779f46a6a7e3182fa6d5a16a12a1
I20260806 12:29:11.064165 12598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:11.070482 12598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f: Bootstrap starting.
I20260806 12:29:11.073108 12598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:11.073838 12598 log.cc:824] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:11.073972 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:11.065404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.6.152.190:39845)
0806 12:29:11.066070 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:11.066086 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:11.066121 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:29:11.066876 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:11.066895 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:11.067351 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:11.067685 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:11.068171 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:11.069157 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:11.070084 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:11.070545 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:11.070719 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:11.071919 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:11.071932 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:11.071951 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:11.072638 (+ 687us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:11.072694 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:11.072715 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:11.072831 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:11.072948 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 12:29:11.073235 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":160,"threads_started":1}
I20260806 12:29:11.076018 12598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f: No bootstrap required, opened a new log
I20260806 12:29:11.076963 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:29:11.080164 12598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e59153daba2445898194eb8d270932f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39845 } }
I20260806 12:29:11.080368 12598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:11.080452 12598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e59153daba2445898194eb8d270932f, State: Initialized, Role: FOLLOWER
I20260806 12:29:11.080868 12598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e59153daba2445898194eb8d270932f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39845 } }
I20260806 12:29:11.081022 12598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:11.081115 12598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:11.081235 12598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:11.082095 12598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e59153daba2445898194eb8d270932f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39845 } }
I20260806 12:29:11.082436 12598 leader_election.cc:302] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e59153daba2445898194eb8d270932f; no voters:
I20260806 12:29:11.082733 12598 leader_election.cc:288] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:11.082885 12603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:11.083120 12603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [term 1 LEADER]: Becoming Leader. State: Replica: 7e59153daba2445898194eb8d270932f, State: Running, Role: LEADER
I20260806 12:29:11.083438 12603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e59153daba2445898194eb8d270932f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39845 } }
I20260806 12:29:11.084044 12598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:11.085038 12604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e59153daba2445898194eb8d270932f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e59153daba2445898194eb8d270932f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39845 } } }
I20260806 12:29:11.085180 12604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:11.085521 12605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e59153daba2445898194eb8d270932f. Latest consensus state: current_term: 1 leader_uuid: "7e59153daba2445898194eb8d270932f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e59153daba2445898194eb8d270932f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39845 } } }
I20260806 12:29:11.085640 12605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:11.085598 12612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:11.088835 12612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:11.094168 12612 catalog_manager.cc:1347] Generated new cluster ID: 934e2234b7a94221b0738722eac6186b
I20260806 12:29:11.094233 12612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:11.104729 12612 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:11.105564 12612 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:11.110231 12612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e59153daba2445898194eb8d270932f: Generated new TSK 0
I20260806 12:29:11.110821 12612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:12.061991 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:11.126469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.6.152.190:39845)
0806 12:29:11.126681 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:11.126686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:11.127176 (+ 490us) server_negotiation.cc:407] Connection header received
0806 12:29:11.127376 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:11.127379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:11.127455 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:11.127569 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0806 12:29:11.127912 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:11.128768 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:11.129644 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:11.129983 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:11.130091 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:11.130357 (+ 266us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 12:29:12.061492 (+931135us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 12:29:12.061832 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59096: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":130}
W20260806 12:29:12.062100 12601 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59096: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 12:29:12.062182 12629 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:11.126350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:39845 (local address 127.0.0.1:59096)
0806 12:29:11.126994 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:11.127010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:11.127158 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:11.127586 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:11.127592 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:11.127634 (+ 42us) client_negotiation.cc:189] Negotiated authn=JWT
0806 12:29:11.127789 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:11.127797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:11.128914 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:11.128917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:11.129533 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:11.129540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:11.130103 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:11.130106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:11.130170 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:11.130192 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 12:29:12.061745 (+931553us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 12:29:12.061950 (+ 205us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.152.190:39845: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":114,"threads_started":1}
I20260806 12:29:12.062647 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12526
2026-08-06T12:29:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1215 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (125 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:13 dist-test-slave-1xrh krb5kdc[12641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:13 dist-test-slave-1xrh krb5kdc[12641](info): set up 2 sockets
Aug 06 12:29:13 dist-test-slave-1xrh krb5kdc[12641](info): commencing operation
krb5kdc: starting...
W20260806 12:29:14.463550 6754 mini_kdc.cc:121] Time spent starting KDC: real 2.126s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:14 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:14.587347 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35615
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:14.697250 12654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:14.697474 12654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:14.697510 12654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:14.701269 12654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:14.701318 12654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:14.701336 12654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:14.701351 12654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:14.701365 12654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:14.706364 12654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:14.707404 12654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:14.708365 12654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:14.716624 12662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:14.716657 12660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:14.716719 12659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:14.716832 12654 server_base.cc:1034] running on GCE node
I20260806 12:29:14.717415 12654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:14.718448 12654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:14.719643 12654 hybrid_clock.cc:648] HybridClock initialized: now 1786019354719625 us; error 29 us; skew 500 ppm
Aug 06 12:29:14 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019354, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:14.725934 12654 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:14.726858 12654 webserver.cc:460] Webserver started at http://127.6.152.190:34867/ using document root <none> and password file <none>
I20260806 12:29:14.727476 12654 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:14.727568 12654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:14.727787 12654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:14.729705 12654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "e19093aca7394db99442d16ccbf6905b"
format_stamp: "Formatted at 2026-08-06 12:29:14 on dist-test-slave-1xrh"
server_key: "ee75df45a33a1ed1f7639e0446b9681a"
server_key_iv: "d15ac90e99ba76074562d984cc89b627"
server_key_version: "encryptionkey@0"
I20260806 12:29:14.730309 12654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "e19093aca7394db99442d16ccbf6905b"
format_stamp: "Formatted at 2026-08-06 12:29:14 on dist-test-slave-1xrh"
server_key: "ee75df45a33a1ed1f7639e0446b9681a"
server_key_iv: "d15ac90e99ba76074562d984cc89b627"
server_key_version: "encryptionkey@0"
I20260806 12:29:14.733914 12654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 12:29:14.736291 12669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:14.737406 12654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:14.737555 12654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "e19093aca7394db99442d16ccbf6905b"
format_stamp: "Formatted at 2026-08-06 12:29:14 on dist-test-slave-1xrh"
server_key: "ee75df45a33a1ed1f7639e0446b9681a"
server_key_iv: "d15ac90e99ba76074562d984cc89b627"
server_key_version: "encryptionkey@0"
I20260806 12:29:14.737659 12654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:14.768953 12654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:14.772526 12654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:14.772742 12654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:14.780146 12654 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:35615
I20260806 12:29:14.780158 12725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:35615 every 8 connection(s)
I20260806 12:29:14.781102 12654 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:14.783391 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12654
I20260806 12:29:14.783537 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:14.783870 6754 external_mini_cluster.cc:1428] Setting key c45ff56f891034fbdd49b42e6c934230
I20260806 12:29:14.783996 12726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:14.790975 12726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b: Bootstrap starting.
Aug 06 12:29:14 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:14.793706 12726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:14.794441 12726 log.cc:824] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:14.796689 12726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b: No bootstrap required, opened a new log
I20260806 12:29:14.798995 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:14.785264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.6.152.190:35615)
0806 12:29:14.785665 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:14.785682 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:14.785717 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:29:14.786587 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:14.786605 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:14.786939 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:14.787292 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:14.788239 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:14.789216 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:14.790403 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:14.790853 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:14.790983 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:14.793822 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:14.793829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:14.793847 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:14.793881 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:14.795894 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:14.796328 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:14.796334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:14.796340 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:14.796491 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:14.796758 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:14.796764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:14.796765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:14.797257 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:14.797451 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:14.797732 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 12:29:14.798017 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":136,"threads_started":1}
I20260806 12:29:14.801007 12726 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19093aca7394db99442d16ccbf6905b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35615 } }
I20260806 12:29:14.801297 12726 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:14.801417 12726 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19093aca7394db99442d16ccbf6905b, State: Initialized, Role: FOLLOWER
I20260806 12:29:14.801821 12726 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19093aca7394db99442d16ccbf6905b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35615 } }
I20260806 12:29:14.801967 12726 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:14.802055 12726 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:14.802153 12726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:14.803001 12726 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19093aca7394db99442d16ccbf6905b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35615 } }
I20260806 12:29:14.803344 12726 leader_election.cc:302] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19093aca7394db99442d16ccbf6905b; no voters:
I20260806 12:29:14.803638 12726 leader_election.cc:288] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:14.803785 12731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:14.804069 12731 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [term 1 LEADER]: Becoming Leader. State: Replica: e19093aca7394db99442d16ccbf6905b, State: Running, Role: LEADER
I20260806 12:29:14.804356 12731 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19093aca7394db99442d16ccbf6905b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35615 } }
I20260806 12:29:14.804827 12726 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:14.806010 12732 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e19093aca7394db99442d16ccbf6905b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19093aca7394db99442d16ccbf6905b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35615 } } }
I20260806 12:29:14.806051 12733 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19093aca7394db99442d16ccbf6905b. Latest consensus state: current_term: 1 leader_uuid: "e19093aca7394db99442d16ccbf6905b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19093aca7394db99442d16ccbf6905b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35615 } } }
I20260806 12:29:14.806141 12732 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:14.806156 12733 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:14.806546 12740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:14.809588 12740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:14.815060 12740 catalog_manager.cc:1347] Generated new cluster ID: 249db51f3870405da1c42736bc4c8785
I20260806 12:29:14.815160 12740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:14.825999 12740 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:14.826946 12740 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:14.834445 12740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e19093aca7394db99442d16ccbf6905b: Generated new TSK 0
I20260806 12:29:14.835201 12740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:29:14.873939 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35615
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:14.983201 12753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:14.983431 12753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:14.983472 12753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:14.987486 12753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:14.987533 12753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:14.987623 12753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:14.992575 12753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:14.993503 12753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:14.994498 12753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:15.003670 12758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:15.003686 12759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:15.003906 12753 server_base.cc:1034] running on GCE node
W20260806 12:29:15.003669 12761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:15.004487 12753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:15.005084 12753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:15.006314 12753 hybrid_clock.cc:648] HybridClock initialized: now 1786019355006301 us; error 26 us; skew 500 ppm
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.008949 12753 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:29:15.009755 12753 webserver.cc:460] Webserver started at http://127.6.152.129:35599/ using document root <none> and password file <none>
I20260806 12:29:15.010275 12753 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:15.010334 12753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:15.010488 12753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:15.012472 12753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "349efd9cdc12416a9361774e7f531df1"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "05315acc8a672e176d4fa528fab0df7e"
server_key_iv: "d05547baa4f15787f69de1025bd5796e"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.012985 12753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "349efd9cdc12416a9361774e7f531df1"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "05315acc8a672e176d4fa528fab0df7e"
server_key_iv: "d05547baa4f15787f69de1025bd5796e"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.016521 12753 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 12:29:15.018707 12769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.019948 12753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:15.020089 12753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "349efd9cdc12416a9361774e7f531df1"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "05315acc8a672e176d4fa528fab0df7e"
server_key_iv: "d05547baa4f15787f69de1025bd5796e"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.020198 12753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:15.034240 12753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:15.037462 12753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:15.037658 12753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:15.038210 12753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:15.038980 12753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:15.039026 12753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.039090 12753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:15.039144 12753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.047704 12753 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:46157
I20260806 12:29:15.047772 12888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:46157 every 8 connection(s)
I20260806 12:29:15.048590 12753 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:15.050213 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12753
I20260806 12:29:15.050351 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:15.050688 6754 external_mini_cluster.cc:1428] Setting key 2f1b70e6a04d043d47658f02d09af554
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:29:15.063019 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.050329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:47445 (local address 127.6.152.190:35615)
0806 12:29:15.050478 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.050483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.051459 (+ 976us) server_negotiation.cc:407] Connection header received
0806 12:29:15.053135 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.053142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.053206 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.053300 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.054582 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.055436 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.056543 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.056772 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.056881 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.059344 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:15.059348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:15.059351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:15.059380 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:15.061094 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.061539 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.061542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.061543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.061618 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.061952 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.061954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.061955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.062211 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:15.062298 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:15.062789 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 12:29:15.062892 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:29:15.063946 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.050774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35615 (local address 127.6.152.129:47445)
0806 12:29:15.051289 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:15.051342 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:15.052894 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:15.053493 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:15.053501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:15.053951 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:15.054428 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:15.054441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.055551 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.055554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:15.056395 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:15.056413 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.056933 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.056937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:15.057097 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.057695 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:15.057711 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:15.059114 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:15.061202 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:15.061207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:15.061217 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:15.061446 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:15.061717 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:15.061719 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:15.061721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:15.061844 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:15.062330 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:15.062338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:15.062580 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 12:29:15.062737 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 12:29:15.062979 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:29:15.065276 12889 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35615
I20260806 12:29:15.065543 12889 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:15.066035 12889 heartbeater.cc:511] Master 127.6.152.190:35615 requested a full tablet report, sending...
I20260806 12:29:15.067787 12686 ts_manager.cc:194] Registered new tserver with Master: 349efd9cdc12416a9361774e7f531df1 (127.6.152.129:46157)
I20260806 12:29:15.069505 12686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:47445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:15.082284 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35615
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:15.212695 12894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:15.212936 12894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:15.213012 12894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:15.217041 12894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:15.217092 12894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:15.217195 12894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:15.222261 12894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:15.223320 12894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:15.224402 12894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:15.233834 12899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:15.233834 12900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:15.233852 12902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:15.234230 12894 server_base.cc:1034] running on GCE node
I20260806 12:29:15.234756 12894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:15.235460 12894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:15.236637 12894 hybrid_clock.cc:648] HybridClock initialized: now 1786019355236613 us; error 44 us; skew 500 ppm
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.239352 12894 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:15.240156 12894 webserver.cc:460] Webserver started at http://127.6.152.130:35815/ using document root <none> and password file <none>
I20260806 12:29:15.240665 12894 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:15.240722 12894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:15.240960 12894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:15.242905 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "719b922179bd4f05b66257de0403b9c0"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "bb452628581a34aec5636cf28f9e0346"
server_key_iv: "43ad2908943691671d81d13af87f9f7b"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.243503 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "719b922179bd4f05b66257de0403b9c0"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "bb452628581a34aec5636cf28f9e0346"
server_key_iv: "43ad2908943691671d81d13af87f9f7b"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.246852 12894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:29:15.249199 12909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.250308 12894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:15.250428 12894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "719b922179bd4f05b66257de0403b9c0"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "bb452628581a34aec5636cf28f9e0346"
server_key_iv: "43ad2908943691671d81d13af87f9f7b"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.250519 12894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:15.270741 12894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:15.273420 12894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:15.273608 12894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:15.274142 12894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:15.274930 12894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:15.275000 12894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.275067 12894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:15.275115 12894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.283303 12894 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:34809
I20260806 12:29:15.283380 13027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:34809 every 8 connection(s)
I20260806 12:29:15.284152 12894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:15.288406 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 12894
I20260806 12:29:15.288527 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:15.288815 6754 external_mini_cluster.cc:1428] Setting key 916f0c0272301e84ef4946d8a5b4296c
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:15.297828 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.285840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:33823 (local address 127.6.152.190:35615)
0806 12:29:15.285967 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.285972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.286680 (+ 708us) server_negotiation.cc:407] Connection header received
0806 12:29:15.288089 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.288105 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.288166 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.288250 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.289563 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.290167 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.291305 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.291541 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.291650 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.294366 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:15.294369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:15.294372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:15.294396 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:15.295826 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.296325 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.296329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.296331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.296412 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.296742 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.296746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.296748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.296938 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:15.297041 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:15.297530 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 12:29:15.297662 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
I20260806 12:29:15.298748 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.286117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35615 (local address 127.6.152.130:33823)
0806 12:29:15.286539 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:15.286579 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:15.287852 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:15.288419 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:15.288425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:15.288836 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:15.289422 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:15.289437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.290279 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.290283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:15.291189 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:15.291199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.291687 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.291691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:15.291928 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.292543 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:15.292568 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:15.294197 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:15.295959 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:15.295964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:15.295979 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:15.296223 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:15.296511 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:15.296514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:15.296516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:15.296649 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:15.297042 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:15.297049 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:15.297297 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 12:29:15.297487 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 12:29:15.297786 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260806 12:29:15.300004 13028 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35615
I20260806 12:29:15.300251 13028 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:15.300786 13028 heartbeater.cc:511] Master 127.6.152.190:35615 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:29:15.301916 12686 ts_manager.cc:194] Registered new tserver with Master: 719b922179bd4f05b66257de0403b9c0 (127.6.152.130:34809)
I20260806 12:29:15.302753 12686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:33823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:15.318984 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35615
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:15.433193 13033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:15.433429 13033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:15.433504 13033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:15.438237 13033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:15.438301 13033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:15.438419 13033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:15.443897 13033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:15.444877 13033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:15.445897 13033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:29:15.455801 13033 server_base.cc:1034] running on GCE node
W20260806 12:29:15.455739 13041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:15.455734 13038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:15.455734 13039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:15.456440 13033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:15.457082 13033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:15.458253 13033 hybrid_clock.cc:648] HybridClock initialized: now 1786019355458239 us; error 24 us; skew 500 ppm
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.460922 13033 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:15.461733 13033 webserver.cc:460] Webserver started at http://127.6.152.131:40763/ using document root <none> and password file <none>
I20260806 12:29:15.462244 13033 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:15.462301 13033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:15.462451 13033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:15.464527 13033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "16d213ed7ce640ca9b9505553ceef252"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "0f59dc4eda9932b0a8535f8294cecfd2"
server_key_iv: "9ca7045f2e3c47dd7203334ee698b542"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.465080 13033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "16d213ed7ce640ca9b9505553ceef252"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "0f59dc4eda9932b0a8535f8294cecfd2"
server_key_iv: "9ca7045f2e3c47dd7203334ee698b542"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.468649 13033 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 12:29:15.471074 13049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.472306 13033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:15.472400 13033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "16d213ed7ce640ca9b9505553ceef252"
format_stamp: "Formatted at 2026-08-06 12:29:15 on dist-test-slave-1xrh"
server_key: "0f59dc4eda9932b0a8535f8294cecfd2"
server_key_iv: "9ca7045f2e3c47dd7203334ee698b542"
server_key_version: "encryptionkey@0"
I20260806 12:29:15.472484 13033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:15.481525 13033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:15.484654 13033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:15.484871 13033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:15.485392 13033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:15.486137 13033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:15.486196 13033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.486234 13033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:15.486255 13033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:15.494673 13033 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:40839
I20260806 12:29:15.494696 13165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:40839 every 8 connection(s)
I20260806 12:29:15.495564 13033 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:15.505321 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13033
I20260806 12:29:15.505476 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:15.505841 6754 external_mini_cluster.cc:1428] Setting key 2573f664f0b3189a827975a8bee4e5f8
I20260806 12:29:15.509145 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.497318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:34317 (local address 127.6.152.190:35615)
0806 12:29:15.497450 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.497454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.498232 (+ 778us) server_negotiation.cc:407] Connection header received
0806 12:29:15.499540 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.499543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.499592 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.499665 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.500729 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.501248 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.502298 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.502515 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.502590 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.505041 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:15.505044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:15.505046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:15.505068 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:15.506760 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.507516 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.507519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.507520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.507599 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.507990 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.507993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.507995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.508240 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:15.508350 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:15.508825 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 12:29:15.508955 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:15.509827 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.497658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35615 (local address 127.6.152.131:34317)
0806 12:29:15.498082 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:15.498122 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:15.499350 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:15.499787 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:15.499794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:15.500132 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:15.500570 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:15.500583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.501354 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.501356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:15.502187 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:15.502196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.502637 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.502640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:15.502769 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.503377 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:15.503393 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:15.504896 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:15.507034 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:15.507039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:15.507050 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:15.507422 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:15.507694 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:15.507697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:15.507699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:15.507873 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:15.508350 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:15.508358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:15.508587 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 12:29:15.508762 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 12:29:15.509084 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260806 12:29:15.510829 13166 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35615
I20260806 12:29:15.511057 13166 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:15.511531 13166 heartbeater.cc:511] Master 127.6.152.190:35615 requested a full tablet report, sending...
I20260806 12:29:15.512446 12686 ts_manager.cc:194] Registered new tserver with Master: 16d213ed7ce640ca9b9505553ceef252 (127.6.152.131:40839)
I20260806 12:29:15.513005 12686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:34317
I20260806 12:29:15.519685 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/test-admin.keytab.
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.535880 6754 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 12:29:15.538594 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:15.537274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.6.152.129:46157)
0806 12:29:15.537540 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.537564 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.537588 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:29:15.537640 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.537645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.537812 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.537991 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.538412 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60878: BlockingRecv error: recv got EOF from 127.0.0.1:60878 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
W20260806 12:29:15.538689 6754 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc' of type FILE
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.539713 6754 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.540745 6754 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:29:15.550087 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.541538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.6.152.129:46157)
0806 12:29:15.541689 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.541692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.541910 (+ 218us) server_negotiation.cc:407] Connection header received
0806 12:29:15.541983 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.541985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.542022 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.542083 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.542513 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.543402 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.544233 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.544603 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.544686 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.546861 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:15.546876 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:15.546889 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:15.546924 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:15.548597 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.548967 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.548971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.548973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.549044 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.549261 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.549263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.549264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.549490 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:15.549627 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:15.549810 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 12:29:15.549967 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 12:29:15.553865 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:15.552856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.6.152.129:46157)
0806 12:29:15.552980 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.552984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.553150 (+ 166us) server_negotiation.cc:407] Connection header received
0806 12:29:15.553288 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.553291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.553327 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.553390 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.553752 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60896: BlockingRecv error: recv got EOF from 127.0.0.1:60896 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 12:29:15.554060 6754 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc' of type FILE
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.555099 6754 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.556006 6754 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:29:15.564109 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.556775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.6.152.129:46157)
0806 12:29:15.556908 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.556912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.556926 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:15.557030 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.557032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.557064 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.557127 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.557570 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.558051 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.558783 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.558992 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.559065 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.561157 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:15.561160 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:15.561161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:15.561182 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:15.562620 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.562976 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.562978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.562979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.563050 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.563341 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.563343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.563344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.563504 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:15.563600 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:15.563813 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 12:29:15.563938 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 12:29:15.566957 13173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:15.565780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.6.152.129:46157)
0806 12:29:15.565893 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.565897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.566048 (+ 151us) server_negotiation.cc:407] Connection header received
0806 12:29:15.566163 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.566166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.566207 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.566290 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.566851 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60912: BlockingRecv error: recv got EOF from 127.0.0.1:60912 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 12:29:15.567092 6754 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786019324359312-6754-0/krb5kdc/krb5cc' of type FILE
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.568105 6754 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:15.568964 6754 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[12641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:29:15.577375 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:15.569756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.6.152.129:46157)
0806 12:29:15.569884 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:15.569890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:15.569963 (+ 73us) server_negotiation.cc:407] Connection header received
0806 12:29:15.570113 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:15.570116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:15.570159 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:15.570237 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:15.570725 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.571277 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.572058 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:15.572283 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:15.572389 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:15.574554 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:15.574557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:15.574559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:15.574581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:15.575880 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.576248 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.576250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.576251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.576320 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:15.576558 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:15.576560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:15.576561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:15.576753 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:15.576851 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:15.577093 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 12:29:15.577216 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:29:15.578254 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12753
I20260806 12:29:15.587818 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12894
I20260806 12:29:15.594275 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13033
I20260806 12:29:15.600364 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 12654
2026-08-06T12:29:15Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3285 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[13186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[13186](info): set up 2 sockets
Aug 06 12:29:15 dist-test-slave-1xrh krb5kdc[13186](info): commencing operation
krb5kdc: starting...
W20260806 12:29:16.663686 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:16 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab.
Entry for principal oryx/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab.
I20260806 12:29:16.786350 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35853
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab
--principal=oryx/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:16.894212 13199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:16.894495 13199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:16.894568 13199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:16.898527 13199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:16.898614 13199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:16.898651 13199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:16.898687 13199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:16.898725 13199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:16.903846 13199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab
--principal=oryx/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:16.904837 13199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:16.905831 13199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:16.914036 13205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:16.914072 13204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:16.914074 13207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:16.914285 13199 server_base.cc:1034] running on GCE node
I20260806 12:29:16.914889 13199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:16.915998 13199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:16.917188 13199 hybrid_clock.cc:648] HybridClock initialized: now 1786019356917170 us; error 29 us; skew 500 ppm
Aug 06 12:29:16 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:16.920059 13199 init.cc:377] Logged in from keytab as oryx/127.6.152.190@KRBTEST.COM (short username oryx)
I20260806 12:29:16.920939 13199 webserver.cc:460] Webserver started at http://127.6.152.190:35371/ using document root <none> and password file <none>
I20260806 12:29:16.921487 13199 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:16.921554 13199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:16.921715 13199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:16.923744 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "b7f89a154cc04ef48cc7d43d56d5f845"
format_stamp: "Formatted at 2026-08-06 12:29:16 on dist-test-slave-1xrh"
server_key: "48f8b7346dd4f8965f9d39256d88cdd9"
server_key_iv: "6beac1eabb63ea6b55065e6020817e39"
server_key_version: "encryptionkey@0"
I20260806 12:29:16.924273 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "b7f89a154cc04ef48cc7d43d56d5f845"
format_stamp: "Formatted at 2026-08-06 12:29:16 on dist-test-slave-1xrh"
server_key: "48f8b7346dd4f8965f9d39256d88cdd9"
server_key_iv: "6beac1eabb63ea6b55065e6020817e39"
server_key_version: "encryptionkey@0"
I20260806 12:29:16.927901 13199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:16.930131 13214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:16.931419 13199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:16.931561 13199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "b7f89a154cc04ef48cc7d43d56d5f845"
format_stamp: "Formatted at 2026-08-06 12:29:16 on dist-test-slave-1xrh"
server_key: "48f8b7346dd4f8965f9d39256d88cdd9"
server_key_iv: "6beac1eabb63ea6b55065e6020817e39"
server_key_version: "encryptionkey@0"
I20260806 12:29:16.931677 13199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:16.968956 13199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:16.971825 13199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:16.972033 13199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:16.978961 13199 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:35853
I20260806 12:29:16.979274 13269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:35853 every 8 connection(s)
I20260806 12:29:16.980003 13199 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:16.982554 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13199
I20260806 12:29:16.982710 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:16.982812 13270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:16.983067 6754 external_mini_cluster.cc:1428] Setting key 62d29d1e47fed2bc75b7130f47a2e7f3
I20260806 12:29:16.989820 13270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Bootstrap starting.
Aug 06 12:29:16 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:16.992664 13270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:16.993384 13270 log.cc:824] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:16.995471 13270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: No bootstrap required, opened a new log
I20260806 12:29:16.997509 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:16.984455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57822 (local address 127.6.152.190:35853)
0806 12:29:16.984854 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:16.984869 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:16.984905 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:29:16.985544 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:16.985561 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:16.985886 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:16.986160 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:16.987161 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:16.988159 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:16.989345 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:16.989802 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:16.989910 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:16.992735 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:16.992740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:16.992753 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:16.992806 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:16.994696 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:16.995198 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:16.995204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:16.995208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:16.995319 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:16.995668 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:16.995671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:16.995672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:16.996048 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:16.996248 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:16.996548 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 12:29:16.996828 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":126,"threads_started":1}
I20260806 12:29:16.999225 13270 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } }
I20260806 12:29:16.999493 13270 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:16.999562 13270 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f89a154cc04ef48cc7d43d56d5f845, State: Initialized, Role: FOLLOWER
I20260806 12:29:16.999974 13270 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } }
I20260806 12:29:17.000097 13270 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:17.000185 13270 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:17.000284 13270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:17.001190 13270 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } }
I20260806 12:29:17.001592 13270 leader_election.cc:302] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7f89a154cc04ef48cc7d43d56d5f845; no voters:
I20260806 12:29:17.001943 13270 leader_election.cc:288] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:17.002053 13275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:17.002305 13275 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 LEADER]: Becoming Leader. State: Replica: b7f89a154cc04ef48cc7d43d56d5f845, State: Running, Role: LEADER
I20260806 12:29:17.002660 13275 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } }
I20260806 12:29:17.003192 13270 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:17.004349 13277 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7f89a154cc04ef48cc7d43d56d5f845. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } }
I20260806 12:29:17.004478 13277 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:17.004841 13276 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } }
I20260806 12:29:17.005028 13276 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:17.005573 13284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:17.008299 13284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:17.013447 13284 catalog_manager.cc:1347] Generated new cluster ID: d2e58b97f0624335b35407b3613d2fc6
I20260806 12:29:17.013517 13284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:17.041052 13284 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:17.041914 13284 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:17.046036 13284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Generated new TSK 0
I20260806 12:29:17.046792 13284 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab.
Entry for principal oryx/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab.
I20260806 12:29:17.086551 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35853
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:17.212613 13298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:17.212857 13298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:17.212931 13298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:17.216909 13298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:17.216960 13298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:17.217067 13298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:17.222154 13298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:17.223177 13298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:17.224162 13298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:17.233608 13306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.233618 13303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.233651 13304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:17.233943 13298 server_base.cc:1034] running on GCE node
I20260806 12:29:17.234369 13298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:17.234965 13298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:17.236151 13298 hybrid_clock.cc:648] HybridClock initialized: now 1786019357236138 us; error 40 us; skew 500 ppm
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:17.238773 13298 init.cc:377] Logged in from keytab as oryx/127.6.152.129@KRBTEST.COM (short username oryx)
I20260806 12:29:17.239622 13298 webserver.cc:460] Webserver started at http://127.6.152.129:37575/ using document root <none> and password file <none>
I20260806 12:29:17.240098 13298 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:17.240155 13298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:17.240307 13298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:17.242139 13298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "d212ad55ae604ab68626673c7c547db2"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "78b00b904975883039f841cc6406e649"
server_key_iv: "2ece02e2fd632be4d6db64aa359a21cb"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.242655 13298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "d212ad55ae604ab68626673c7c547db2"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "78b00b904975883039f841cc6406e649"
server_key_iv: "2ece02e2fd632be4d6db64aa359a21cb"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.246121 13298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:29:17.248402 13313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.249519 13298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:17.249658 13298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "d212ad55ae604ab68626673c7c547db2"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "78b00b904975883039f841cc6406e649"
server_key_iv: "2ece02e2fd632be4d6db64aa359a21cb"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.249754 13298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:17.264052 13298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:17.266812 13298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:17.267002 13298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:17.267582 13298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:17.268344 13298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:17.268390 13298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.268457 13298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:17.268489 13298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.277134 13298 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:39471
I20260806 12:29:17.277170 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:39471 every 8 connection(s)
I20260806 12:29:17.277977 13298 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:17.283066 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13298
I20260806 12:29:17.283253 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:17.283540 6754 external_mini_cluster.cc:1428] Setting key 529a21ba635fa21a13d26be64e2ccc63
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:17.292989 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.279854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:54555 (local address 127.6.152.190:35853)
0806 12:29:17.279995 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:17.279999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:17.280946 (+ 947us) server_negotiation.cc:407] Connection header received
0806 12:29:17.281735 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:17.281738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:17.281784 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:17.281857 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:17.283064 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.283782 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.285221 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.285449 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.285531 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.288605 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:17.288608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:17.288611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:17.288634 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:17.290448 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.291160 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.291167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.291172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.291261 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.291717 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.291720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.291721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.291971 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:17.292052 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:17.292689 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 12:29:17.292848 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:29:17.294114 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.280161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:54555)
0806 12:29:17.280795 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:17.280835 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:17.281561 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:17.282022 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:17.282028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:17.282369 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:17.282882 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.282903 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.283935 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.283939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.285103 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.285115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.285610 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.285617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.285845 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.286462 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:17.286488 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:17.288428 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:17.290576 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.290585 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.290602 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.290996 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.291379 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.291385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.291388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.291593 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.292096 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:17.292105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:17.292409 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 12:29:17.292646 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 12:29:17.293022 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for oryx/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:29:17.295459 13432 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35853
I20260806 12:29:17.295697 13432 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:17.296162 13432 heartbeater.cc:511] Master 127.6.152.190:35853 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.130@KRBTEST.COM" created.
I20260806 12:29:17.297638 13233 ts_manager.cc:194] Registered new tserver with Master: d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471)
I20260806 12:29:17.299356 13233 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:54555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab.
Entry for principal oryx/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab.
I20260806 12:29:17.313932 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35853
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:17.423709 13437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:17.423966 13437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:17.424007 13437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:17.427853 13437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:17.427898 13437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:17.427953 13437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:17.433120 13437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:17.434042 13437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:17.435099 13437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:17.444597 13445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.444640 13442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.444614 13443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:17.444792 13437 server_base.cc:1034] running on GCE node
I20260806 12:29:17.445266 13437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:17.445910 13437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:17.447099 13437 hybrid_clock.cc:648] HybridClock initialized: now 1786019357447091 us; error 30 us; skew 500 ppm
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:17.449944 13437 init.cc:377] Logged in from keytab as oryx/127.6.152.130@KRBTEST.COM (short username oryx)
I20260806 12:29:17.450736 13437 webserver.cc:460] Webserver started at http://127.6.152.130:44047/ using document root <none> and password file <none>
I20260806 12:29:17.451303 13437 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:17.451370 13437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:17.451530 13437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:17.453471 13437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "b0c6a1b3ad5643d29e070caba1f17fe0"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "868a76a4ee9d01f30864434d72f690e7"
server_key_iv: "19b868e9ad1fb11416b9c0291296e300"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.454061 13437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "b0c6a1b3ad5643d29e070caba1f17fe0"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "868a76a4ee9d01f30864434d72f690e7"
server_key_iv: "19b868e9ad1fb11416b9c0291296e300"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.457526 13437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 12:29:17.459867 13452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.461014 13437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:17.461104 13437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "b0c6a1b3ad5643d29e070caba1f17fe0"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "868a76a4ee9d01f30864434d72f690e7"
server_key_iv: "19b868e9ad1fb11416b9c0291296e300"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.461179 13437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:17.478041 13437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:17.481274 13437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:17.481408 13437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:17.481892 13437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:17.482604 13437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:17.482646 13437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.482683 13437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:17.482705 13437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.491217 13437 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:38753
I20260806 12:29:17.491230 13568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:38753 every 8 connection(s)
I20260806 12:29:17.492053 13437 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:17.500604 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13437
I20260806 12:29:17.500739 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:17.501034 6754 external_mini_cluster.cc:1428] Setting key aca05c8ec4b72bd9224e696758dcbacd
I20260806 12:29:17.505432 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.493692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:58487 (local address 127.6.152.190:35853)
0806 12:29:17.493867 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:17.493872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:17.494855 (+ 983us) server_negotiation.cc:407] Connection header received
0806 12:29:17.495749 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:17.495752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:17.495798 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:17.495881 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:17.496956 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.497591 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.498640 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.498853 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.498927 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.501292 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:17.501295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:17.501297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:17.501326 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:17.502887 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.503449 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.503453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.503454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.503534 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.503982 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.503988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.503993 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.504279 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:17.504395 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:17.505095 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 12:29:17.505225 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 12:29:17.506318 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.494103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:58487)
0806 12:29:17.494711 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:17.494749 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:17.495570 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:17.496023 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:17.496030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:17.496346 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:17.496814 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.496826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.497695 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.497697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.498538 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.498546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.498975 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.498977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.499104 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.499594 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:17.499609 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:17.501099 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:17.503001 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.503007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.503020 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.503345 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.503676 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.503678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.503680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.503856 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.504441 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:17.504449 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:17.504792 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 12:29:17.505028 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 12:29:17.505382 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":148,"threads_started":1}
I20260806 12:29:17.507506 13569 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35853
I20260806 12:29:17.507747 13569 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:17.508230 13569 heartbeater.cc:511] Master 127.6.152.190:35853 requested a full tablet report, sending...
I20260806 12:29:17.509193 13233 ts_manager.cc:194] Registered new tserver with Master: b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753)
I20260806 12:29:17.509748 13233 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:58487
WARNING: no policy specified for oryx/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab.
Entry for principal oryx/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab.
I20260806 12:29:17.530150 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35853
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:17.638453 13574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:17.638679 13574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:17.638720 13574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:17.642835 13574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:17.642889 13574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:17.642994 13574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:17.648101 13574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:17.649041 13574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:17.649953 13574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:17.659252 13582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.659382 13580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.659253 13579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:17.659492 13574 server_base.cc:1034] running on GCE node
I20260806 12:29:17.659893 13574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:17.660542 13574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:17.661710 13574 hybrid_clock.cc:648] HybridClock initialized: now 1786019357661695 us; error 26 us; skew 500 ppm
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:17.664577 13574 init.cc:377] Logged in from keytab as oryx/127.6.152.131@KRBTEST.COM (short username oryx)
I20260806 12:29:17.665536 13574 webserver.cc:460] Webserver started at http://127.6.152.131:41339/ using document root <none> and password file <none>
I20260806 12:29:17.666100 13574 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:17.666160 13574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:17.666400 13574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:17.668460 13574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "779b158c765e480291319a80e639c1d3"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "ca89216a9d05397698ee736a85921c3d"
server_key_iv: "6818bb0a5b7b8e6298a8d6799314d4fd"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.669029 13574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "779b158c765e480291319a80e639c1d3"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "ca89216a9d05397698ee736a85921c3d"
server_key_iv: "6818bb0a5b7b8e6298a8d6799314d4fd"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.672596 13574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:17.674965 13589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.676129 13574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:17.676259 13574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "779b158c765e480291319a80e639c1d3"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "ca89216a9d05397698ee736a85921c3d"
server_key_iv: "6818bb0a5b7b8e6298a8d6799314d4fd"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.676360 13574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:17.709808 13574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:17.712823 13574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:17.712975 13574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:17.713480 13574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:17.714287 13574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:17.714359 13574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.714409 13574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:17.714432 13574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.723551 13574 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:34491
I20260806 12:29:17.723577 13712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:34491 every 8 connection(s)
I20260806 12:29:17.724426 13574 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:17.726011 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13574
I20260806 12:29:17.726161 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:17.726462 6754 external_mini_cluster.cc:1428] Setting key e0a30b40b72f135cb2c45940afb83617
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:17.737690 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.727276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58285 (local address 127.6.152.190:35853)
0806 12:29:17.727426 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:17.727430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:17.727704 (+ 274us) server_negotiation.cc:407] Connection header received
0806 12:29:17.728776 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:17.728779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:17.728834 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:17.728913 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:17.729946 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.730557 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.731673 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.731873 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.731949 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.734284 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:17.734287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:17.734289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:17.734310 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:17.735717 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.736190 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.736193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.736194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.736263 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.736547 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.736550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.736551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.736705 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:17.736777 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:17.737364 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 12:29:17.737506 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:29:17.738583 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.726675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:58285)
0806 12:29:17.727545 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:17.727603 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:17.728546 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:17.729071 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:17.729077 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:17.729395 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:17.729828 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.729840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.730662 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.730665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.731571 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.731580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.732002 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.732005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.732182 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.732717 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:17.732731 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:17.734144 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:17.735858 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.735862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.735872 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.736105 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.736346 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.736348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.736350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.736470 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.736786 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:17.736790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:17.737082 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 12:29:17.737311 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 12:29:17.737668 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":182,"threads_started":1}
I20260806 12:29:17.739846 13713 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35853
I20260806 12:29:17.740101 13713 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:17.740548 13713 heartbeater.cc:511] Master 127.6.152.190:35853 requested a full tablet report, sending...
I20260806 12:29:17.741551 13233 ts_manager.cc:194] Registered new tserver with Master: 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491)
I20260806 12:29:17.742101 13233 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:58285
I20260806 12:29:17.752569 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.152.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.189.keytab.
Entry for principal oryx/127.6.152.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.189.keytab.
I20260806 12:29:17.780236 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:44319
--webserver_interface=127.6.152.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.189.keytab
--principal=oryx/127.6.152.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35853,127.6.152.189:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:17.889209 13718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:17.889457 13718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:17.889525 13718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:17.894001 13718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:17.894085 13718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:17.894127 13718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:17.894167 13718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:17.894209 13718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:17.902154 13718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35853,127.6.152.189:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.189.keytab
--principal=oryx/127.6.152.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.152.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
W20260806 12:29:17.902319 13718 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.152.190:35853,127.6.152.189:44319'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 12:29:17.903055 13718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:17.904080 13718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:17.911888 13726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:17.912071 13718 server_base.cc:1034] running on GCE node
W20260806 12:29:17.911900 13724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:17.912019 13723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:17.912835 13718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:17.913811 13718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:17.915011 13718 hybrid_clock.cc:648] HybridClock initialized: now 1786019357914993 us; error 29 us; skew 500 ppm
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:17.917654 13718 init.cc:377] Logged in from keytab as oryx/127.6.152.189@KRBTEST.COM (short username oryx)
I20260806 12:29:17.918411 13718 webserver.cc:460] Webserver started at http://127.6.152.189:44993/ using document root <none> and password file <none>
I20260806 12:29:17.918900 13718 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:17.918968 13718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:17.919240 13718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:17.921133 13718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/instance:
uuid: "54894143a9974e90b1f1aa1a145d4efd"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "3dffc83dc4042e0c15cefb01801791af"
server_key_iv: "cda8dd02c18c30bf61114b3ea525d738"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.921609 13718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal/instance:
uuid: "54894143a9974e90b1f1aa1a145d4efd"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "3dffc83dc4042e0c15cefb01801791af"
server_key_iv: "cda8dd02c18c30bf61114b3ea525d738"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.925295 13718 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:17.927546 13734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:17.928721 13718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:17.928826 13718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
uuid: "54894143a9974e90b1f1aa1a145d4efd"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "3dffc83dc4042e0c15cefb01801791af"
server_key_iv: "cda8dd02c18c30bf61114b3ea525d738"
server_key_version: "encryptionkey@0"
I20260806 12:29:17.928912 13718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:17.947503 13718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:17.950086 13718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:17.950238 13718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:17.958339 13718 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.189:44319
I20260806 12:29:17.958601 13788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.189:44319 every 8 connection(s)
I20260806 12:29:17.959342 13718 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
I20260806 12:29:17.961880 13790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:17.965631 13790 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:29:17.966411 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13718
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.189@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:17.979648 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.967628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.6.152.190:35853)
0806 12:29:17.967749 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:17.967756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:17.968658 (+ 902us) server_negotiation.cc:407] Connection header received
0806 12:29:17.969415 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:17.969417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:17.969468 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:17.969543 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:17.970506 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.971015 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.972165 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.972414 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.972495 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.975394 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:17.975397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:17.975399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:17.975431 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:17.977004 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.978058 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.978061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.978062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.978137 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.978475 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.978477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.978479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.978709 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:17.978831 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:17.979378 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 12:29:17.979503 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:29:17.980636 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.967829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:57830)
0806 12:29:17.968442 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:17.968487 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:17.969195 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:17.969613 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:17.969618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:17.970075 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:17.970339 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.970358 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.971172 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.971175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.972038 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.972047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.972556 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.972560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.972726 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.973488 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:17.973504 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:17.975193 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:17.977139 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.977149 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.977160 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.977638 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.978229 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.978235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.978240 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.978385 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.978826 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:17.978832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:17.978988 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 12:29:17.979308 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 12:29:17.979676 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":62,"spinlock_wait_cycles":55040,"thread_start_us":176,"threads_started":1}
I20260806 12:29:17.980636 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.967829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.6.152.189:44319)
0806 12:29:17.968338 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:17.968358 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:17.968396 (+ 38us) server_negotiation.cc:407] Connection header received
0806 12:29:17.969193 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:17.969215 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:17.969449 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:17.969643 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:17.970380 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.971373 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.972211 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.972621 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.972731 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.975763 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:17.975769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:17.975778 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:17.975800 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:17.977648 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.978053 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.978057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.978058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.978155 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.978412 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.978418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.978423 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.978938 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:17.979193 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:17.979874 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 12:29:17.980019 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":342,"thread_start_us":160,"threads_started":1}
I20260806 12:29:17.982239 13790 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:29:17 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.189@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:17.989791 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.982856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.6.152.189:44319)
0806 12:29:17.982957 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:17.982963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:17.982976 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:17.983101 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:17.983104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:17.983165 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:17.983240 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:17.983604 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.984099 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.984798 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.985033 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.985138 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.986976 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:17.986979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:17.986981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:17.987003 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:17.988291 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.988684 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.988691 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.988692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.988768 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:17.989041 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:17.989044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:17.989046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:17.989200 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:17.989300 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:17.989532 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 12:29:17.989658 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":45}
I20260806 12:29:17.989791 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:17.982797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:43468)
0806 12:29:17.982885 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:17.982901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:17.983004 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:17.983262 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:17.983264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:17.983408 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:17.983487 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.983492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.984203 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.984208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.984688 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:17.984695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:17.985135 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:17.985138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:17.985259 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:17.985695 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:17.985707 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:17.986832 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:17.988398 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.988404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.988406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.988591 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.988864 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:17.988866 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:17.988868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:17.988949 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:17.989290 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:17.989293 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:17.989368 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 12:29:17.989477 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 12:29:17.989610 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":6}
I20260806 12:29:17.993261 13790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Bootstrap starting.
I20260806 12:29:17.995419 13790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:17.996068 13790 log.cc:824] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:17.997762 13790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: No bootstrap required, opened a new log
I20260806 12:29:18.000578 13790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } }
I20260806 12:29:18.000771 13790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:18.000802 13790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54894143a9974e90b1f1aa1a145d4efd, State: Initialized, Role: FOLLOWER
I20260806 12:29:18.001145 13790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } }
I20260806 12:29:18.001577 13796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } } }
I20260806 12:29:18.001708 13796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:18.002229 13790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: configured and running, proceeding with master startup.
W20260806 12:29:18.007056 13807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:29:18.007161 13807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:29:18.010166 13718 master_runner.cc:428] Detected that this master 54894143a9974e90b1f1aa1a145d4efd is joining an existing cluster
I20260806 12:29:18.010246 13718 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.152.189:44319
I20260806 12:29:18.020288 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.012642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43474 (local address 127.6.152.189:44319)
0806 12:29:18.012750 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.012756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.012769 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:18.012815 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.012817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.012851 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.012927 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.013543 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.014278 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.015593 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.015857 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.015962 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.016896 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.016899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.016902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.016923 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.018643 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.019076 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.019083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.019085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.019182 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.019476 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.019480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.019482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.019646 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.019751 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.020019 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.020161 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":40}
I20260806 12:29:18.020303 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.012260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:43474)
0806 12:29:18.012633 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.012646 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.012739 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.012969 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.012972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.013237 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.013415 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.013424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.014440 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.014444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.015449 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.015460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.015963 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.015966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.016048 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.016452 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.016462 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.016764 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.018758 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.018764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.018769 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.018974 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.019288 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.019296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.019297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.019368 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.019741 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.019744 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.019831 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.019964 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.020102 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":48,"thread_start_us":108,"threads_started":1}
I20260806 12:29:18.021565 13813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.012222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:57846)
0806 12:29:18.012531 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.012546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.012688 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.013055 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.013057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.013274 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.013399 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.013406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.014460 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.014462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.015108 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.015160 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.015680 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.015684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.015772 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.016825 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.016840 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.017327 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.019656 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.019660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.019662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.019930 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.020248 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.020251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.020253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.020334 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.021056 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.021059 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.021151 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.021255 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.021389 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":57,"thread_start_us":127,"threads_started":1}
I20260806 12:29:18.021620 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.012285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57846 (local address 127.6.152.190:35853)
0806 12:29:18.012429 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.012434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.012616 (+ 182us) server_negotiation.cc:407] Connection header received
0806 12:29:18.012805 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.012808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.012853 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.012921 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.013524 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.014311 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.015276 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.015540 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.015664 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.017501 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.017505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.017508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.017535 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.019528 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.020049 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.020053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.020055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.020137 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.020693 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.020698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.020700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.020933 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.021050 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.021342 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.021475 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.190@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:18.036141 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.026138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:43482)
0806 12:29:18.026521 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.026574 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.026703 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.026985 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.026991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.027440 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.027565 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.027573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.028236 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.028240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.029227 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.029239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.029663 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.029667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.029789 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.030248 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.030263 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.032005 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.034049 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.034060 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.034066 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.034401 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.034721 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.034727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.034729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.034825 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.035378 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.035387 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.035530 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.035684 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.035895 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":112,"threads_started":1}
I20260806 12:29:18.036662 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.026185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.6.152.189:44319)
0806 12:29:18.026290 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.026294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.026657 (+ 363us) server_negotiation.cc:407] Connection header received
0806 12:29:18.026822 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.026824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.026863 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.026925 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.027697 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.028133 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.029350 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.029548 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.029621 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.032142 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.032150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.032152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.032172 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.033921 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.034519 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.034525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.034530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.034613 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.034939 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.034945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.034949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.035209 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.035319 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.036470 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.036554 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:29:18.038008 13233 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.152.189:44319: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd"
member_type: NON_VOTER
last_known_addr {
host: "127.6.152.189"
port: 44319
}
attrs {
promote: true
}
}
dest_uuid: "b7f89a154cc04ef48cc7d43d56d5f845"
cas_config_opid_index: -1
I20260806 12:29:18.038643 13233 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } }
I20260806 12:29:18.040980 13762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:18.042219 13762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Refusing update from remote peer b7f89a154cc04ef48cc7d43d56d5f845: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 12:29:18.040498 13818 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } }
I20260806 12:29:18.043257 13819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
WARNING: no policy specified for oryx/127.6.152.188@KRBTEST.COM; defaulting to no policy
I20260806 12:29:18.044224 13818 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.152.189:44319
I20260806 12:29:18.044761 13718 master.cc:537] Master@127.6.152.189:44319 shutting down...
I20260806 12:29:18.045439 13818 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } }
I20260806 12:29:18.045619 13818 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
W20260806 12:29:18.045606 13216 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.188@KRBTEST.COM" created.
W20260806 12:29:18.047540 13216 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 -> Peer 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189:44319): Couldn't send request to peer 54894143a9974e90b1f1aa1a145d4efd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 12:29:18.049973 13718 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:18.050264 13718 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:18.050402 13718 tablet_replica.cc:330] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.188.keytab.
Entry for principal oryx/127.6.152.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.188.keytab.
I20260806 12:29:18.065959 13718 master.cc:552] Master@127.6.152.189:44319 shutdown complete.
I20260806 12:29:18.066017 13718 master_runner.cc:305] Clearing existing system tablet
I20260806 12:29:18.066293 13718 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.066361 13718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.066533 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:35583
--webserver_interface=127.6.152.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.188.keytab
--principal=oryx/127.6.152.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:29:18.067806 13718 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:18.068746 13827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.069029 13718 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.069303 13718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:18.069434 13718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
uuid: "54894143a9974e90b1f1aa1a145d4efd"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "3dffc83dc4042e0c15cefb01801791af"
server_key_iv: "cda8dd02c18c30bf61114b3ea525d738"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.071538 13718 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:18.074026 13718 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 12:29:18.074132 13718 log.cc:1196] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 12:29:18.074431 13718 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Deleting consensus metadata
I20260806 12:29:18.074601 13718 master_runner.cc:315] Copying system tablet from 127.6.152.190:35853
I20260806 12:29:18.076089 13718 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: tablet copy: Beginning tablet copy session from remote peer at address 127.6.152.190:35853
I20260806 12:29:18.088122 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.076883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57850 (local address 127.6.152.190:35853)
0806 12:29:18.077010 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.077015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.079532 (+ 2517us) server_negotiation.cc:407] Connection header received
0806 12:29:18.079773 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.079778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.079838 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.079935 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.080467 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.081313 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.082474 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.082836 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.082956 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.084364 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.084368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.084371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.084400 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.086140 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.086646 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.086650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.086653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.086744 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.087056 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.087064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.087067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.087335 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.087445 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.087788 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.087917 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:29:18.088693 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.076812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:57850)
0806 12:29:18.079371 (+ 2559us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.079391 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.079639 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.079983 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.079986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.080216 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.080359 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.080366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.081431 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.081434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.082349 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.082360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.082972 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.082976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.083077 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.083733 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.083748 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.084198 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.086266 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.086270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.086272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.086537 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.086845 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.086852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.086855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.086951 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.087497 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.087500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.087602 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.088365 (+ 763us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.088515 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2435,"thread_start_us":94,"threads_started":1}
I20260806 12:29:18.089484 13254 tablet_copy_service.cc:140] P b7f89a154cc04ef48cc7d43d56d5f845: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 54894143a9974e90b1f1aa1a145d4efd ({username='oryx', principal='oryx/127.6.152.189@KRBTEST.COM'} at 127.0.0.1:57850)
I20260806 12:29:18.089681 13254 tablet_copy_service.cc:161] P b7f89a154cc04ef48cc7d43d56d5f845: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 54894143a9974e90b1f1aa1a145d4efd at {username='oryx', principal='oryx/127.6.152.189@KRBTEST.COM'} at 127.0.0.1:57850: session id = 54894143a9974e90b1f1aa1a145d4efd-00000000000000000000000000000000
I20260806 12:29:18.091715 13254 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 12:29:18.093678 13718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 12:29:18.096253 13718 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: tablet copy: Starting download of 0 data blocks...
I20260806 12:29:18.096462 13718 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: tablet copy: Starting download of 1 WAL segments...
I20260806 12:29:18.098394 13718 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 12:29:18.100256 13254 tablet_copy_service.cc:342] P b7f89a154cc04ef48cc7d43d56d5f845: Request end of tablet copy session 54894143a9974e90b1f1aa1a145d4efd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.152.189@KRBTEST.COM'} at 127.0.0.1:57850
I20260806 12:29:18.100378 13254 tablet_copy_service.cc:433] P b7f89a154cc04ef48cc7d43d56d5f845: ending tablet copy session 54894143a9974e90b1f1aa1a145d4efd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 54894143a9974e90b1f1aa1a145d4efd
I20260806 12:29:18.108701 13718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:18.110464 13838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.110445 13835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.111006 13836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.119678 13718 server_base.cc:1034] running on GCE node
I20260806 12:29:18.119858 13718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:18.120164 13718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:18.121409 13718 hybrid_clock.cc:648] HybridClock initialized: now 1786019358121384 us; error 45 us; skew 500 ppm
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:18.123081 13718 init.cc:377] Logged in from keytab as oryx/127.6.152.189@KRBTEST.COM (short username oryx)
I20260806 12:29:18.123606 13718 webserver.cc:460] Webserver started at http://127.6.152.189:39223/ using document root <none> and password file <none>
I20260806 12:29:18.123808 13718 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.123855 13718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.125159 13718 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:29:18.126206 13845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.126475 13718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:29:18.126554 13718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
uuid: "54894143a9974e90b1f1aa1a145d4efd"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "3dffc83dc4042e0c15cefb01801791af"
server_key_iv: "cda8dd02c18c30bf61114b3ea525d738"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.126637 13718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.144980 13718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:18.147244 13718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:18.163581 13899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.189:44319 every 8 connection(s)
I20260806 12:29:18.163620 13718 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.189:44319
I20260806 12:29:18.164429 13718 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
I20260806 12:29:18.167069 13900 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:29:18.168377 13900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Bootstrap starting.
I20260806 12:29:18.185973 13900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:29:18.186278 13900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: Bootstrap complete.
I20260806 12:29:18.186942 13900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } }
I20260806 12:29:18.187086 13900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54894143a9974e90b1f1aa1a145d4efd, State: Initialized, Role: LEARNER
I20260806 12:29:18.187227 13900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } }
I20260806 12:29:18.187623 13903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } }
I20260806 12:29:18.187737 13900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:18.187860 13903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: LEARNER
I20260806 12:29:18.192660 13915 catalog_manager.cc:1259] Loaded cluster ID: d2e58b97f0624335b35407b3613d2fc6
I20260806 12:29:18.192781 13915 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: loading cluster ID for follower catalog manager: success
I20260806 12:29:18.194960 13915 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: acquiring CA information for follower catalog manager: success
I20260806 12:29:18.196256 13915 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 12:29:18.218017 13826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:18.218292 13826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:18.218370 13826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:18.222457 13826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:18.222543 13826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:18.222580 13826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:18.222635 13826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:18.222669 13826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:18.228137 13826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.188.keytab
--principal=oryx/127.6.152.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:35583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.152.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:18.229079 13826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:18.230093 13826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:18.239287 13922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.239420 13924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.239277 13921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.239810 13826 server_base.cc:1034] running on GCE node
I20260806 12:29:18.240281 13826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:18.241235 13826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:18.242425 13826 hybrid_clock.cc:648] HybridClock initialized: now 1786019358242396 us; error 41 us; skew 500 ppm
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:18.245432 13826 init.cc:377] Logged in from keytab as oryx/127.6.152.188@KRBTEST.COM (short username oryx)
I20260806 12:29:18.246258 13826 webserver.cc:460] Webserver started at http://127.6.152.188:41261/ using document root <none> and password file <none>
I20260806 12:29:18.246799 13826 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.246889 13826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.247169 13826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:18.249184 13826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/instance:
uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
format_stamp: "Formatted at 2026-08-06 12:29:18 on dist-test-slave-1xrh"
server_key: "f2f1956a57f2c34b0cd60b908d814761"
server_key_iv: "4aa5c6f5a4f001c8ca0e1a5fa9dbb510"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.249761 13826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal/instance:
uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
format_stamp: "Formatted at 2026-08-06 12:29:18 on dist-test-slave-1xrh"
server_key: "f2f1956a57f2c34b0cd60b908d814761"
server_key_iv: "4aa5c6f5a4f001c8ca0e1a5fa9dbb510"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.253748 13826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 12:29:18.256309 13933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.257517 13826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:18.257655 13826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
format_stamp: "Formatted at 2026-08-06 12:29:18 on dist-test-slave-1xrh"
server_key: "f2f1956a57f2c34b0cd60b908d814761"
server_key_iv: "4aa5c6f5a4f001c8ca0e1a5fa9dbb510"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.257763 13826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.276387 13826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:18.280085 13826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:18.280282 13826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:18.297035 13826 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.188:35583
I20260806 12:29:18.297423 13986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.188:35583 every 8 connection(s)
I20260806 12:29:18.298137 13826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
I20260806 12:29:18.301234 13987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:18.301486 13432 heartbeater.cc:503] Master 127.6.152.190:35853 was elected leader, sending a full tablet report...
I20260806 12:29:18.305809 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 13826
I20260806 12:29:18.306553 13987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.188@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.152.188@KRBTEST.COM
I20260806 12:29:18.320492 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.307333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.6.152.188:35583)
0806 12:29:18.307812 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.307849 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.307881 (+ 32us) server_negotiation.cc:407] Connection header received
0806 12:29:18.309113 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.309132 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.309446 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.309894 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.310692 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.311862 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.312701 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.313172 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.313297 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.316316 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.316322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.316332 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.316353 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.318201 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.318654 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.318662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.318665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.318753 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.319066 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.319068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.319069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.319407 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.319666 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.319841 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.319993 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":160,"threads_started":1}
I20260806 12:29:18.320501 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.308543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:57856)
0806 12:29:18.309218 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.309264 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.309603 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.309927 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.309932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.310300 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.310642 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.310656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.311615 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.311618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.312504 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.312514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.313004 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.313008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.313201 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.313891 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.313915 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.315578 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.317449 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.317457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.317465 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.317802 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.318119 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.318125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.318127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.318297 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.318715 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.318720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.319024 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.319265 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.319553 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":323,"threads_started":1}
I20260806 12:29:18.321652 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.308676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.6.152.190:35853)
0806 12:29:18.308858 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.308863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.309513 (+ 650us) server_negotiation.cc:407] Connection header received
0806 12:29:18.309723 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.309726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.309777 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.309880 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.310788 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.311399 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.312631 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.312856 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.312969 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.315733 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.315738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.315741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.315775 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.317322 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.317926 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.317930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.317932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.318017 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.318389 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.318391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.318392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.318600 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.318710 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.321445 (+ 2735us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.321544 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 12:29:18.322669 13987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.188@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:18.330331 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.323232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:43498)
0806 12:29:18.323335 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.323350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.323424 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.323879 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.323881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.324025 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.324110 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.324115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.324984 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.324986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.325489 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.325496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.325891 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.325894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.326018 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.326378 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.326388 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.327705 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.329137 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.329140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.329141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.329319 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.329580 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.329582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.329584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.329651 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.329997 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.329999 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.330066 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.330141 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.330230 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 12:29:18.330384 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.323336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.6.152.189:44319)
0806 12:29:18.323608 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.323611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.323623 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:29:18.323659 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.323661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.323699 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.323848 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.324225 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.324892 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.325584 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.325801 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.325875 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.327830 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.327833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.327836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.327858 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.329041 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.329405 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.329407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.329409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.329481 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.329728 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.329730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.329732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.329889 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.329995 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.330177 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.330269 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":115,"threads_started":1}
I20260806 12:29:18.331317 13987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.188@KRBTEST.COM for oryx/127.6.152.188@KRBTEST.COM
I20260806 12:29:18.338524 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.331730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:42276)
0806 12:29:18.331836 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.331851 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.331919 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.332088 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.332090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.332206 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.332266 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.332270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.333043 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.333045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.333585 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.333594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.334048 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.334051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.334110 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.334455 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.334465 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.335726 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.337341 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.337343 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.337344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.337517 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.337761 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.337763 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.337764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.337830 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.338154 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.338156 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.338223 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.338306 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.338400 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 12:29:18.338598 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.331817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.6.152.188:35583)
0806 12:29:18.331924 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.331928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.331968 (+ 40us) server_negotiation.cc:407] Connection header received
0806 12:29:18.331995 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.331997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.332027 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.332081 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.332353 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.332868 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.333693 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.333930 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.334008 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.335845 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.335848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.335850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.335870 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.337250 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.337617 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.337619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.337620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.337685 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.337901 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.337903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.337904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.338056 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.338145 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.338335 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.338453 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":40}
I20260806 12:29:18.342499 13987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Bootstrap starting.
I20260806 12:29:18.345506 13987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:18.346674 13987 log.cc:824] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:18.348762 13987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: No bootstrap required, opened a new log
I20260806 12:29:18.352682 13987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } }
I20260806 12:29:18.352897 13987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:18.352981 13987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e65e23e4cec4adcb3b40b1ffefaf192, State: Initialized, Role: FOLLOWER
I20260806 12:29:18.353408 13987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } }
I20260806 12:29:18.354056 13993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } } }
I20260806 12:29:18.354197 13993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:18.354815 13987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 12:29:18.361816 14004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:29:18.361920 14004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:29:18.365748 13826 master_runner.cc:428] Detected that this master 1e65e23e4cec4adcb3b40b1ffefaf192 is joining an existing cluster
I20260806 12:29:18.365828 13826 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.152.188:35583
I20260806 12:29:18.374203 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13298
I20260806 12:29:18.383749 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:39471
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:35853
--webserver_port=37575
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:29:18.396270 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.370468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.6.152.188:35583)
0806 12:29:18.370604 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.370608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.371736 (+ 1128us) server_negotiation.cc:407] Connection header received
0806 12:29:18.371898 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.371901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.371957 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.372038 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.372928 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.373613 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.375988 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.376342 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.376444 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.391982 (+ 15538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.391986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.391989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.392013 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.393755 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.394643 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.394646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.394648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.394786 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.395133 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.395138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.395141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.395445 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.395544 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.395978 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.396101 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":68}
I20260806 12:29:18.396991 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.370729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:42292)
0806 12:29:18.371411 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.371425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.371792 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.372167 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.372172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.372489 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.372802 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.372810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.373724 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.373727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.375547 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.375558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.376656 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.376660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.376798 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.377590 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.377611 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.391720 (+ 14109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.394205 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.394210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.394215 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.394502 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.394910 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.394914 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.394917 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.395010 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.395599 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.395604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.395706 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.396622 (+ 916us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.396757 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":389,"spinlock_wait_cycles":768,"thread_start_us":231,"threads_started":1}
I20260806 12:29:18.398775 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.369309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.6.152.190:35853)
0806 12:29:18.369436 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.369443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.370267 (+ 824us) server_negotiation.cc:407] Connection header received
0806 12:29:18.370548 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.370552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.370605 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.370659 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.372302 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.373200 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.375380 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.375732 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.375844 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.393274 (+ 17430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.393280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.393286 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.393335 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.395145 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.396508 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.396512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.396514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.396601 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.397184 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.397187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.397189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.397412 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.397492 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.398499 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.398623 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:29:18.399583 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.369355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:57868)
0806 12:29:18.370008 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.370024 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.370394 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.370803 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.370806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.371533 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.372096 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.372108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.373358 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.373361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.375188 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.375203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.377123 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.377128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.377458 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.378105 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.378121 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.393082 (+ 14961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.396085 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.396088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.396090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.396328 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.396705 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.396707 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.396709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.396787 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.397670 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.397673 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.397769 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.399261 (+ 1492us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.399411 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":786,"thread_start_us":288,"threads_started":1}
I20260806 12:29:18.400894 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.369693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:43506)
0806 12:29:18.370260 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.370275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.370646 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.371146 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.371152 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.371774 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.372087 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.372097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.373270 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.373273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.375372 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.375387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.376953 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.376960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.377114 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.391663 (+ 14549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.391702 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.392445 (+ 743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.396864 (+ 4419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.396867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.396869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.397076 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.398247 (+ 1171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.398250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.398252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.398345 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.400421 (+ 2076us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.400424 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.400509 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.400601 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.400725 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":781,"spinlock_wait_cycles":7552,"thread_start_us":257,"threads_started":1}
I20260806 12:29:18.400986 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.369355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.6.152.189:44319)
0806 12:29:18.369502 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.369507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.370577 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 12:29:18.370779 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.370782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.370872 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.370955 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.372217 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.373142 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.375680 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.376128 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.376219 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.392617 (+ 16398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.392621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.392624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.392654 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.395895 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.398026 (+ 2131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.398030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.398032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.398127 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.399927 (+ 1800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.399930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.399933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.400270 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.400378 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.400697 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.400832 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019356, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.190@KRBTEST.COM for oryx/127.6.152.188@KRBTEST.COM
I20260806 12:29:18.412796 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.403781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:42302)
0806 12:29:18.404012 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.404026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.404126 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.404422 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.404424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.404619 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.404717 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.404723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.405699 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.405702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.406568 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.406578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.407142 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.407146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.407239 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.407714 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.407729 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.409215 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.411096 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.411100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.411101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.411401 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.411694 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.411697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.411699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.411799 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.412290 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.412293 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.412407 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.412511 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.412647 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 12:29:18.412828 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.403881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.6.152.188:35583)
0806 12:29:18.404014 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.404019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.404073 (+ 54us) server_negotiation.cc:407] Connection header received
0806 12:29:18.404245 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.404248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.404310 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.404393 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.404831 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.405575 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.406688 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.407008 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.407110 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.409361 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.409364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.409367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.409391 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.410977 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.411503 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.411507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.411509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.411591 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.411931 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.411935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.411937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.412168 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.412283 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.412581 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.412691 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 12:29:18.414227 13233 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.152.188:35583: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
member_type: NON_VOTER
last_known_addr {
host: "127.6.152.188"
port: 35583
}
attrs {
promote: true
}
}
dest_uuid: "b7f89a154cc04ef48cc7d43d56d5f845"
cas_config_opid_index: 5
I20260806 12:29:18.414872 13233 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } }
I20260806 12:29:18.416011 13821 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.416652 13956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:18.417475 13821 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.152.188:35583
I20260806 12:29:18.417660 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.417749 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:18.418038 13956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Refusing update from remote peer b7f89a154cc04ef48cc7d43d56d5f845: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 12:29:18.418530 13826 master.cc:537] Master@127.6.152.188:35583 shutting down...
I20260806 12:29:18.418725 13820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 12:29:18.420902 13216 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 -> Peer 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188:35583): Couldn't send request to peer 1e65e23e4cec4adcb3b40b1ffefaf192. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 12:29:18.424481 13826 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:18.424742 13826 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:18.424878 13826 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: stopping tablet replica
I20260806 12:29:18.428985 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.416105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:43520)
0806 12:29:18.422005 (+ 5900us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.422021 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.422103 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.422516 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.422520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.422536 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:18.422615 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.422621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.424731 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.424734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.426033 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.426045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.428538 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.428541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.428661 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.428719 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.428796 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.428864 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5827}
I20260806 12:29:18.429406 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.418012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.6.152.189:44319)
0806 12:29:18.419467 (+ 1455us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.419472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.422186 (+ 2714us) server_negotiation.cc:407] Connection header received
0806 12:29:18.422240 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.422243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.422306 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.422397 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:18.423311 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.424510 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.426179 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.427280 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.428239 (+ 959us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.428396 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.429209 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.429280 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1259}
I20260806 12:29:18.429418 13826 master.cc:552] Master@127.6.152.188:35583 shutdown complete.
I20260806 12:29:18.429456 13826 master_runner.cc:305] Clearing existing system tablet
I20260806 12:29:18.429762 13826 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.429980 13826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.430107 13872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 LEARNER]: Refusing update from remote peer b7f89a154cc04ef48cc7d43d56d5f845: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 12:29:18.430586 13820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 12:29:18.432466 13872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.434409 13903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7f89a154cc04ef48cc7d43d56d5f845. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.434507 13903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: LEARNER
I20260806 12:29:18.434614 13903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: NON_VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: true } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.434715 13903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: LEARNER
I20260806 12:29:18.435387 13820 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: attempting to promote NON_VOTER 54894143a9974e90b1f1aa1a145d4efd to VOTER
I20260806 12:29:18.435815 13820 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } }
I20260806 12:29:18.437207 13872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 LEARNER]: Refusing update from remote peer b7f89a154cc04ef48cc7d43d56d5f845: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 12:29:18.437642 13821 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 12:29:18.437906 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.436839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:42314)
0806 12:29:18.437696 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.437836 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":795}
I20260806 12:29:18.438670 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.438430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:42328)
0806 12:29:18.438560 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.438608 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:35583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 12:29:18.438870 13216 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 -> Peer 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188:35583): Couldn't send request to peer 1e65e23e4cec4adcb3b40b1ffefaf192. Status: Network error: Client connection negotiation failed: client connection to 127.6.152.188:35583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 12:29:18.439278 13819 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.440734 13819 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.440826 13819 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:18.440936 13819 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.440996 13819 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:18.441540 13872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.442946 13826 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.000s sys 0.000s
I20260806 12:29:18.443518 14016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.443692 14016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:18.443868 14017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.444067 13826 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.444195 13826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 12:29:18.444231 13826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
format_stamp: "Formatted at 2026-08-06 12:29:18 on dist-test-slave-1xrh"
server_key: "f2f1956a57f2c34b0cd60b908d814761"
server_key_iv: "4aa5c6f5a4f001c8ca0e1a5fa9dbb510"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.446560 13826 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:18.448741 13826 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 12:29:18.448830 13826 log.cc:1196] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 12:29:18.449070 13826 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Deleting consensus metadata
I20260806 12:29:18.449223 13826 master_runner.cc:315] Copying system tablet from 127.6.152.190:35853
I20260806 12:29:18.450574 13826 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: tablet copy: Beginning tablet copy session from remote peer at address 127.6.152.190:35853
I20260806 12:29:18.462692 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.451052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:57876)
0806 12:29:18.454595 (+ 3543us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.454611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.454770 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.455045 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.455048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.455303 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.455447 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.455456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.456553 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.456557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.457472 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.457483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.458138 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.458142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.458238 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.458835 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.458850 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.459274 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.461126 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.461129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.461131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.461374 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.461663 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.461665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.461667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.461751 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.462169 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.462172 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.462266 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.462371 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.462498 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3466,"thread_start_us":162,"threads_started":1}
I20260806 12:29:18.462721 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.451163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.6.152.190:35853)
0806 12:29:18.451271 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.451276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.454720 (+ 3444us) server_negotiation.cc:407] Connection header received
0806 12:29:18.454891 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.454894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.454947 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.455024 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.455623 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.456416 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.457637 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.458023 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.458129 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.459419 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.459423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.459426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.459452 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.461016 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.461482 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.461489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.461491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.461575 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.461849 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.461852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.461854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.462076 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.462168 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.462455 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.462568 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:29:18.463586 13254 tablet_copy_service.cc:140] P b7f89a154cc04ef48cc7d43d56d5f845: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1e65e23e4cec4adcb3b40b1ffefaf192 ({username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:57876)
I20260806 12:29:18.463711 13254 tablet_copy_service.cc:161] P b7f89a154cc04ef48cc7d43d56d5f845: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1e65e23e4cec4adcb3b40b1ffefaf192 at {username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:57876: session id = 1e65e23e4cec4adcb3b40b1ffefaf192-00000000000000000000000000000000
I20260806 12:29:18.464582 13254 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 12:29:18.465860 13826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 12:29:18.468303 13826 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: tablet copy: Starting download of 0 data blocks...
I20260806 12:29:18.468506 13826 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: tablet copy: Starting download of 1 WAL segments...
I20260806 12:29:18.470059 13826 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 12:29:18.471796 13254 tablet_copy_service.cc:342] P b7f89a154cc04ef48cc7d43d56d5f845: Request end of tablet copy session 1e65e23e4cec4adcb3b40b1ffefaf192-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:57876
I20260806 12:29:18.471879 13254 tablet_copy_service.cc:433] P b7f89a154cc04ef48cc7d43d56d5f845: ending tablet copy session 1e65e23e4cec4adcb3b40b1ffefaf192-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1e65e23e4cec4adcb3b40b1ffefaf192
I20260806 12:29:18.479877 13826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:18.488283 14025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.488380 14024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.488572 13826 server_base.cc:1034] running on GCE node
W20260806 12:29:18.488854 14027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.489059 13826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:18.489375 13826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:18.490525 13826 hybrid_clock.cc:648] HybridClock initialized: now 1786019358490498 us; error 46 us; skew 500 ppm
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:18.496237 13826 init.cc:377] Logged in from keytab as oryx/127.6.152.188@KRBTEST.COM (short username oryx)
I20260806 12:29:18.496721 13826 webserver.cc:460] Webserver started at http://127.6.152.188:42093/ using document root <none> and password file <none>
I20260806 12:29:18.496933 13826 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.496979 13826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.498416 13826 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:18.499485 14034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.499761 13826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:18.499838 13826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
format_stamp: "Formatted at 2026-08-06 12:29:18 on dist-test-slave-1xrh"
server_key: "f2f1956a57f2c34b0cd60b908d814761"
server_key_iv: "4aa5c6f5a4f001c8ca0e1a5fa9dbb510"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.499917 13826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.511739 13569 heartbeater.cc:503] Master 127.6.152.190:35853 was elected leader, sending a full tablet report...
I20260806 12:29:18.519903 13826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:18.522164 13826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:18.527549 13826 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.188:35583
I20260806 12:29:18.527742 14093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.188:35583 every 8 connection(s)
I20260806 12:29:18.528133 13826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
I20260806 12:29:18.529940 14094 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:29:18.530972 14094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Bootstrap starting.
W20260806 12:29:18.531375 14013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:18.531682 14013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:18.531761 14013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:18.537716 14013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:18.537781 14013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:18.537868 14013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:18.541761 14094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:29:18.542028 14094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Bootstrap complete.
I20260806 12:29:18.542574 14094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } }
I20260806 12:29:18.542702 14094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1e65e23e4cec4adcb3b40b1ffefaf192, State: Initialized, Role: LEARNER
I20260806 12:29:18.542814 14094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } }
I20260806 12:29:18.543239 14097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.543340 14094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:18.543447 14097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: This master's current role is: LEARNER
I20260806 12:29:18.545676 14013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:39471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=37575
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:18.546806 14013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:18.547856 14013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:29:18.548163 14108 catalog_manager.cc:1259] Loaded cluster ID: d2e58b97f0624335b35407b3613d2fc6
I20260806 12:29:18.548280 14108 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: loading cluster ID for follower catalog manager: success
I20260806 12:29:18.551281 14108 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: acquiring CA information for follower catalog manager: success
I20260806 12:29:18.552837 14108 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 12:29:18.557489 14114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.557536 14110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.557515 14112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.558092 14013 server_base.cc:1034] running on GCE node
I20260806 12:29:18.558543 14013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:18.559341 14013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:18.560595 14013 hybrid_clock.cc:648] HybridClock initialized: now 1786019358560562 us; error 31 us; skew 500 ppm
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:18.563953 14013 init.cc:377] Logged in from keytab as oryx/127.6.152.129@KRBTEST.COM (short username oryx)
I20260806 12:29:18.564863 14013 webserver.cc:460] Webserver started at http://127.6.152.129:37575/ using document root <none> and password file <none>
I20260806 12:29:18.565485 14013 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.565585 14013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.569308 14013 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:18.571468 14121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.572602 14013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:18.572731 14013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "d212ad55ae604ab68626673c7c547db2"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "78b00b904975883039f841cc6406e649"
server_key_iv: "2ece02e2fd632be4d6db64aa359a21cb"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.573256 14013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.585713 14013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:18.588869 14013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:18.589109 14013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:18.589986 14013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:18.591025 14013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:18.591101 14013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.591193 14013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:18.591239 14013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.612644 14013 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:39471
I20260806 12:29:18.612820 14238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:39471 every 8 connection(s)
I20260806 12:29:18.613763 14013 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:18.622290 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 14013
I20260806 12:29:18.622531 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13437
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:18.633402 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.616089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:54821 (local address 127.6.152.190:35853)
0806 12:29:18.616232 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.616238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.618093 (+ 1855us) server_negotiation.cc:407] Connection header received
0806 12:29:18.619488 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.619493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.619559 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.619646 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.621337 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.621885 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.624160 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.624516 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.624602 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.628663 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.628666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.628669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.628704 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.630736 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.631374 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.631377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.631379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.631465 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.631893 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.631900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.631912 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.632178 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.632279 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.632993 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.633148 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:29:18.634321 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.616786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:54821)
0806 12:29:18.617702 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.617763 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.619230 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.619838 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.619847 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.620517 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.621169 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.621184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.622142 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.622144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.624016 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.624030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.624806 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.624812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.625123 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.625910 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.625927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.628502 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.630865 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.630871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.630886 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.631254 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.631568 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.631572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.631574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.631753 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.632308 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.632317 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.632649 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.632886 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.633233 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":29,"spinlock_wait_cycles":74496,"thread_start_us":487,"threads_started":1}
I20260806 12:29:18.634382 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:38753
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:35853
--webserver_port=44047
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:29:18.635977 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.616130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:44451 (local address 127.6.152.189:44319)
0806 12:29:18.616239 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.616245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.618041 (+ 1796us) server_negotiation.cc:407] Connection header received
0806 12:29:18.619611 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.619614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.619669 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.619766 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.621380 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.622194 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.628436 (+ 6242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.628879 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.629020 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.631656 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.631660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.631663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.631690 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.633375 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.634087 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.634091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.634093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.634190 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.634934 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.634938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.634939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.635262 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.635343 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.635644 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.635766 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:29:18.636466 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.616786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.6.152.129:44451)
0806 12:29:18.617702 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.617763 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.619230 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.619839 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.619849 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.620587 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.621222 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.621228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.622359 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.622363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.628284 (+ 5921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.628296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.629011 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.629016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.629209 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.629790 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.629809 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.631523 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.633717 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.633720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.633722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.633952 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.634296 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.634299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.634300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.634823 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.635394 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.635396 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.635465 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.635573 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.635729 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":5014,"spinlock_wait_cycles":6144,"thread_start_us":390,"threads_started":1}
I20260806 12:29:18.637598 14241 heartbeater.cc:347] Connected to a master server at 127.6.152.189:44319
I20260806 12:29:18.637619 14240 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35853
I20260806 12:29:18.637895 14241 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:18.637912 14240 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:18.638477 14240 heartbeater.cc:511] Master 127.6.152.190:35853 requested a full tablet report, sending...
I20260806 12:29:18.638475 14241 heartbeater.cc:511] Master 127.6.152.189:44319 requested a full tablet report, sending...
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.188@KRBTEST.COM
I20260806 12:29:18.640024 13862 ts_manager.cc:194] Registered new tserver with Master: d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471)
I20260806 12:29:18.640192 13233 ts_manager.cc:194] Re-registered known tserver with Master: d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471)
I20260806 12:29:18.640990 13233 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:54821
I20260806 12:29:18.643098 14245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.618153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.6.152.129:55331)
0806 12:29:18.618643 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.618655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.619234 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.620205 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.620217 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.620587 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.621217 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.621228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.634598 (+ 13370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.634648 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.636071 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.636087 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.637013 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.637018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.637139 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.637631 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.637647 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.639375 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.641424 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.641427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.641431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.641695 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.642041 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.642044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.642046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.642137 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.642648 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.642652 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.642732 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.642838 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.642976 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":79,"spinlock_wait_cycles":31232,"thread_start_us":185,"threads_started":1}
I20260806 12:29:18.643381 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.618707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:55331 (local address 127.6.152.188:35583)
0806 12:29:18.619037 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.619041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.619054 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:18.619782 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.619785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.619834 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.620169 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.621371 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.634415 (+ 13044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.636293 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.636778 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.636884 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.639549 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.639552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.639555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.639581 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.641295 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.641817 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.641823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.641827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.641932 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.642261 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.642267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.642271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.642534 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.642639 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.642945 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.643104 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":134,"threads_started":1}
I20260806 12:29:18.644068 14239 heartbeater.cc:347] Connected to a master server at 127.6.152.188:35583
I20260806 12:29:18.644170 14239 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:18.644399 14239 heartbeater.cc:511] Master 127.6.152.188:35583 requested a full tablet report, sending...
I20260806 12:29:18.645594 14055 ts_manager.cc:194] Registered new tserver with Master: d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471)
I20260806 12:29:18.743980 13713 heartbeater.cc:503] Master 127.6.152.190:35853 was elected leader, sending a full tablet report...
W20260806 12:29:18.753923 14247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:18.754187 14247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:18.754259 14247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:18.758169 14247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:18.758250 14247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:18.758339 14247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:18.763233 14247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:38753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=44047
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:18.764219 14247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:18.765231 14247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:18.774775 14255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.774863 14252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.774763 14253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.775112 14247 server_base.cc:1034] running on GCE node
I20260806 12:29:18.775514 14247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:18.776165 14247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:18.777362 14247 hybrid_clock.cc:648] HybridClock initialized: now 1786019358777329 us; error 52 us; skew 500 ppm
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:18.780388 14247 init.cc:377] Logged in from keytab as oryx/127.6.152.130@KRBTEST.COM (short username oryx)
I20260806 12:29:18.781224 14247 webserver.cc:460] Webserver started at http://127.6.152.130:44047/ using document root <none> and password file <none>
I20260806 12:29:18.781714 14247 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.781785 14247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.785482 14247 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:18.787444 14263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.788712 14247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:18.788807 14247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "b0c6a1b3ad5643d29e070caba1f17fe0"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "868a76a4ee9d01f30864434d72f690e7"
server_key_iv: "19b868e9ad1fb11416b9c0291296e300"
server_key_version: "encryptionkey@0"
I20260806 12:29:18.789397 14247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:18.816474 14247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:18.819221 14247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:18.819367 14247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:18.819845 14247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:18.820694 14247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:18.820768 14247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.820839 14247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:18.820891 14247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:18.831262 14247 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:38753
I20260806 12:29:18.831306 14379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:38753 every 8 connection(s)
I20260806 12:29:18.832304 14247 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:18.836088 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 14247
I20260806 12:29:18.836357 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13574
I20260806 12:29:18.845357 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:34491
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:35853
--webserver_port=41339
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.188@KRBTEST.COM
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:18.851429 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.834248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:51593 (local address 127.6.152.190:35853)
0806 12:29:18.834459 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.834464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.836427 (+ 1963us) server_negotiation.cc:407] Connection header received
0806 12:29:18.837197 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.837203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.837267 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.837352 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.839454 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.840346 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.842457 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.842843 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.842910 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.846582 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.846587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.846590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.846624 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.848871 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.849540 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.849548 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.849553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.849661 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.850099 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.850104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.850109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.850348 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.850452 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.851092 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.851237 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 12:29:18.851778 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.836175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:48933 (local address 127.6.152.188:35583)
0806 12:29:18.837509 (+ 1334us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.837514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.837527 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:18.837573 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.837579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.837638 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.837717 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.839477 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.840276 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.842565 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.843029 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.843152 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.847612 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.847619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.847624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.847659 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.849517 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.850109 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.850115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.850119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.850214 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.850571 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.850577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.850580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.850854 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.850950 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.851397 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.851547 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1247}
I20260806 12:29:18.852560 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.837327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:38069 (local address 127.6.152.189:44319)
0806 12:29:18.837454 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.837458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.837474 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:29:18.837765 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.837769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.837842 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.837906 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:18.839589 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.840306 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.842457 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.842799 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.842887 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.848550 (+ 5663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:18.848557 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:18.848563 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:18.848596 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:18.850349 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.850845 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.850848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.850850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.850944 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:18.851299 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:18.851302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:18.851304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:18.851517 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:18.851593 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.852309 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.852427 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:29:18.853085 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.834863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:51593)
0806 12:29:18.836129 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.836144 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.837079 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.837647 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.837659 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.838452 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.839279 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.839304 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.840512 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.840518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.842314 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.842325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.843093 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.843097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.843474 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.844582 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.844603 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.846380 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.849013 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.849021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.849037 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.849404 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.849787 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.849793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.849796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.849975 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.850516 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.850523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.850817 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.851647 (+ 830us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.851994 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":274,"spinlock_wait_cycles":131968,"thread_start_us":200,"threads_started":1}
I20260806 12:29:18.853089 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.834863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.6.152.130:48933)
0806 12:29:18.835342 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.835412 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.836830 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.838425 (+ 1595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.838431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.838592 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.839352 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.839363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.840543 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.840546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.842419 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.842433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.843405 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.843409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.843640 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.844445 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.844460 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.847453 (+ 2993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.849652 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.849658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.849663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.849967 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.850342 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.850349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.850353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.850452 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.851019 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.851024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.851170 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.852004 (+ 834us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.852173 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":928,"thread_start_us":136,"threads_started":1}
I20260806 12:29:18.853816 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.835851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.6.152.130:38069)
0806 12:29:18.836348 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.836363 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.837642 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.838079 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.838085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.838893 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:18.839474 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.839479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.840522 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.840525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.842314 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.842325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.843093 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.843097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.843444 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.844306 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:18.844331 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:18.848314 (+ 3983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:18.850471 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.850474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.850476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.850729 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.851058 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:18.851060 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:18.851062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:18.851186 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:18.852043 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:18.852045 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:18.852128 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.853007 (+ 879us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.853148 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":628,"spinlock_wait_cycles":1280,"thread_start_us":78,"threads_started":1}
I20260806 12:29:18.854494 14380 heartbeater.cc:347] Connected to a master server at 127.6.152.188:35583
I20260806 12:29:18.854502 14381 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35853
I20260806 12:29:18.854723 14381 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:18.854724 14380 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:18.855206 14382 heartbeater.cc:347] Connected to a master server at 127.6.152.189:44319
I20260806 12:29:18.855289 14380 heartbeater.cc:511] Master 127.6.152.188:35583 requested a full tablet report, sending...
I20260806 12:29:18.855293 14381 heartbeater.cc:511] Master 127.6.152.190:35853 requested a full tablet report, sending...
I20260806 12:29:18.855295 14382 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:18.855525 14382 heartbeater.cc:511] Master 127.6.152.189:44319 requested a full tablet report, sending...
I20260806 12:29:18.856452 13233 ts_manager.cc:194] Re-registered known tserver with Master: b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753)
I20260806 12:29:18.856621 13862 ts_manager.cc:194] Registered new tserver with Master: b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753)
I20260806 12:29:18.857309 13233 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:51593
I20260806 12:29:18.857954 14055 ts_manager.cc:194] Registered new tserver with Master: b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753)
I20260806 12:29:18.897030 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.893347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.6.152.188:35583)
0806 12:29:18.893526 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:18.893531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:18.893551 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:29:18.893630 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:18.893633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:18.893695 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:18.893768 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:18.894090 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.894639 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.896095 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.896519 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.896604 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.896711 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:18.896862 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 12:29:18.896923 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 12:29:18.897055 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:18.893227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:42332)
0806 12:29:18.893382 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:18.893417 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:18.893527 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:18.893806 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:18.893809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:18.893829 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:18.893937 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.893943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.894759 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.894764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.895977 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:18.895989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:18.896637 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:18.896641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:18.896735 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:18.896762 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 12:29:18.896841 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:29:18.896918 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 12:29:18.897795 14067 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 12:29:18.898152 14097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7f89a154cc04ef48cc7d43d56d5f845. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:18.898363 14097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: This master's current role is: LEARNER
W20260806 12:29:18.963923 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:18.964170 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:18.964243 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:18.968304 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:18.968357 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:18.968413 14387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:18.973687 14387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:34491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=41339
--tserver_master_addrs=127.6.152.190:35853,127.6.152.189:44319,127.6.152.188:35583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:18.974684 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:18.975747 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:18.985711 14394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:18.985726 14396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.985920 14387 server_base.cc:1034] running on GCE node
W20260806 12:29:18.985719 14393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:18.986462 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:18.987206 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:18.988382 14387 hybrid_clock.cc:648] HybridClock initialized: now 1786019358988349 us; error 43 us; skew 500 ppm
Aug 06 12:29:18 dist-test-slave-1xrh krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:18.991724 14387 init.cc:377] Logged in from keytab as oryx/127.6.152.131@KRBTEST.COM (short username oryx)
I20260806 12:29:18.992619 14387 webserver.cc:460] Webserver started at http://127.6.152.131:41339/ using document root <none> and password file <none>
I20260806 12:29:18.993175 14387 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:18.993273 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:18.997115 14387 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 12:29:18.998958 14404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:19.000361 14387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:19.000494 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "779b158c765e480291319a80e639c1d3"
format_stamp: "Formatted at 2026-08-06 12:29:17 on dist-test-slave-1xrh"
server_key: "ca89216a9d05397698ee736a85921c3d"
server_key_iv: "6818bb0a5b7b8e6298a8d6799314d4fd"
server_key_version: "encryptionkey@0"
I20260806 12:29:19.001050 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:19.013962 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:19.017186 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:19.017380 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:19.018077 14387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:19.019008 14387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:19.019061 14387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:19.019129 14387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:19.019188 14387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:19.028860 14387 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:34491
I20260806 12:29:19.028952 14521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:34491 every 8 connection(s)
I20260806 12:29:19.029831 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:19.033504 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 14387
Aug 06 12:29:19 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.188@KRBTEST.COM
Aug 06 12:29:19 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
Aug 06 12:29:19 dist-test-slave-1xrh krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.189@KRBTEST.COM
I20260806 12:29:19.048079 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.033643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:52727 (local address 127.6.152.188:35583)
0806 12:29:19.033795 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.033800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.034046 (+ 246us) server_negotiation.cc:407] Connection header received
0806 12:29:19.034608 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.034614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.034680 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.034738 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.036725 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.037574 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.039569 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.039900 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.039992 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.043700 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.043704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.043707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.043730 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.045603 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.046320 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.046327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.046329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.046420 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.046844 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.046849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.046852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.047072 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.047192 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.047791 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.047928 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:29:19.048753 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.031738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:51877 (local address 127.6.152.189:44319)
0806 12:29:19.031868 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.031872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.033054 (+ 1182us) server_negotiation.cc:407] Connection header received
0806 12:29:19.034538 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.034542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.034599 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.034653 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.036886 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.037636 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.039570 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.039900 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.039985 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.045003 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.045007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.045010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.045045 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.047067 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.047548 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.047551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.047552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.047629 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.047917 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.047920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.047922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.048105 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.048188 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.048526 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.048624 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:29:19.049511 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.032302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.6.152.131:51877)
0806 12:29:19.032892 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.032949 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.034289 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.035044 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.035216 (+ 172us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.035673 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:19.036745 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.036754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.037767 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.037769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.039430 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.039440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.040145 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.040148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.040530 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.041411 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:19.041430 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:19.044843 (+ 3413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:19.047200 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:19.047205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:19.047208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:19.047436 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:19.047735 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:19.047737 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:19.047739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:19.047816 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:19.048244 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:19.048246 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:19.048323 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.048893 (+ 570us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.049019 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":545,"spinlock_wait_cycles":10752,"thread_start_us":142,"threads_started":1}
I20260806 12:29:19.049580 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.033381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.6.152.131:52727)
0806 12:29:19.033902 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.033919 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.034405 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.035045 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.035053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.035669 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:19.036572 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.036584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.037749 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.037753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.039430 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.039440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.040145 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.040148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.040512 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.041510 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:19.041528 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:19.043519 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:19.045779 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:19.045784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:19.045799 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:19.046204 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:19.046545 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:19.046548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:19.046550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:19.046730 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:19.047251 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:19.047259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:19.047534 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.048228 (+ 694us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.048541 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":767,"spinlock_wait_cycles":1408,"thread_start_us":149,"threads_started":1}
I20260806 12:29:19.049760 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.031893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:54511 (local address 127.6.152.190:35853)
0806 12:29:19.031997 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.032001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.033057 (+ 1056us) server_negotiation.cc:407] Connection header received
0806 12:29:19.034538 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.034542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.034599 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.034654 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.036710 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.037534 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.039575 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.039975 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.040087 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.045898 (+ 5811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.045904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.045910 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.045938 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.047813 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.048398 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.048403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.048406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.048492 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.048764 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.048767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.048770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.049015 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.049113 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.049441 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.049549 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 12:29:19.050297 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.032302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:54511)
0806 12:29:19.032892 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.032949 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.034289 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.035044 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.035053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.035707 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:19.036500 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.036522 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.037742 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.037745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.039432 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.039441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.040351 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.040355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.040485 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.041385 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:19.041417 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:19.045712 (+ 4295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:19.047923 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:19.047926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:19.047928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:19.048288 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:19.048591 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:19.048594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:19.048595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:19.048671 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:19.049171 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:19.049173 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:19.049266 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.049953 (+ 687us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.050110 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":335,"spinlock_wait_cycles":125568,"thread_start_us":158,"threads_started":1}
I20260806 12:29:19.051259 14523 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35853
I20260806 12:29:19.051283 14524 heartbeater.cc:347] Connected to a master server at 127.6.152.189:44319
I20260806 12:29:19.051514 14524 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:19.051604 14522 heartbeater.cc:347] Connected to a master server at 127.6.152.188:35583
I20260806 12:29:19.051697 14522 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:19.051517 14523 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:19.052125 14524 heartbeater.cc:511] Master 127.6.152.189:44319 requested a full tablet report, sending...
I20260806 12:29:19.052232 14522 heartbeater.cc:511] Master 127.6.152.188:35583 requested a full tablet report, sending...
I20260806 12:29:19.052156 14523 heartbeater.cc:511] Master 127.6.152.190:35853 requested a full tablet report, sending...
I20260806 12:29:19.053336 13862 ts_manager.cc:194] Registered new tserver with Master: 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491)
I20260806 12:29:19.053349 14055 ts_manager.cc:194] Registered new tserver with Master: 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491)
I20260806 12:29:19.053349 13233 ts_manager.cc:194] Re-registered known tserver with Master: 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491)
I20260806 12:29:19.054121 13233 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:54511
I20260806 12:29:19.058946 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.037505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.6.152.188:35583)
0806 12:29:19.038354 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.038409 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.040381 (+ 1972us) server_negotiation.cc:407] Connection header received
0806 12:29:19.040428 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.040432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.040478 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.040621 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.044103 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.045104 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.048423 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.048753 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.049232 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.054654 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.054657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.054660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.054685 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.056966 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.057441 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.057446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.057451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.057528 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.057813 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.057818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.057823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.058027 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.058141 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.058671 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.058784 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":140,"threads_started":1}
I20260806 12:29:19.059226 14533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.035949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.6.152.189:44319)
0806 12:29:19.038208 (+ 2259us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.038213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.039386 (+ 1173us) server_negotiation.cc:407] Connection header received
0806 12:29:19.039429 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.039432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.039485 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.039762 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.041796 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.042642 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.043961 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.044891 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.045004 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.055401 (+ 10397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.055405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.055407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.055431 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.057266 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.057705 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.057709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.057710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.057790 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.058079 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.058084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.058085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.058267 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.058378 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.058865 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.059000 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2186,"spinlock_wait_cycles":3456,"thread_start_us":98,"threads_started":1}
I20260806 12:29:19.059626 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.036594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.6.152.190:35853)
0806 12:29:19.045682 (+ 9088us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.045687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.045702 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:29:19.045738 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.045743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.045815 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.045941 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.049335 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.052199 (+ 2864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.053692 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.053979 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.054104 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.055943 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.055946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.055949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.055973 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.057685 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.058114 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.058118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.058121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.058194 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.058424 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.058427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.058429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.058654 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.058764 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.059365 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.059478 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9001,"thread_start_us":73,"threads_started":1}
I20260806 12:29:19.063282 13233 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:19.066233 13233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:19.068794 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: NON_VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: true } } }
I20260806 12:29:19.068933 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:19.072420 14540 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: attempting to promote NON_VOTER 1e65e23e4cec4adcb3b40b1ffefaf192 to VOTER
I20260806 12:29:19.072858 14540 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:19.073989 14067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 LEARNER]: Refusing update from remote peer b7f89a154cc04ef48cc7d43d56d5f845: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 12:29:19.074362 13872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Refusing update from remote peer b7f89a154cc04ef48cc7d43d56d5f845: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 12:29:19.074404 14542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 12:29:19.074751 14542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 12:29:19.076540 13821 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.077723 13872 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.078261 14540 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.078298 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.078379 14540 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:19.078395 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:19.079538 14550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.079674 14550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:19.079656 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.079763 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:19.080000 14067 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.082007 14097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:19.082149 14097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:19.089859 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.084028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39471 (local address 127.0.0.1:38856)
0806 12:29:19.084174 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.084190 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.084290 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.085722 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.085725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.085745 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:19.085868 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.085876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.087204 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.087210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.088239 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.088251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.089475 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.089479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.089589 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.089617 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.089664 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.089730 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":1}
I20260806 12:29:19.091006 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.084311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:38753 (local address 127.0.0.1:39056)
0806 12:29:19.085067 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.085082 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.085400 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.086379 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.086381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.086395 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:19.086488 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.086492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.087953 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.087957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.089474 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.089483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.090590 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.090594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.090711 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.090736 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.090789 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.090869 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":74,"thread_start_us":281,"threads_started":1}
I20260806 12:29:19.091396 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.084473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.6.152.129:39471)
0806 12:29:19.084750 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.084769 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.084793 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:29:19.084873 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.084879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.085319 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.085745 (+ 426us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:19.086022 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.087058 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.088388 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.089330 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.090241 (+ 911us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.090421 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.090481 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.090601 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":97,"threads_started":1}
I20260806 12:29:19.091378 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.084731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:34491 (local address 127.0.0.1:38244)
0806 12:29:19.086050 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.086066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.086297 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.087263 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.087266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.087285 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:19.087420 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.087427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.088659 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.088663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.089847 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.089879 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.090807 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.090811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.090902 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.090924 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.091052 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.091248 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"mutex_wait_us":2,"thread_start_us":158,"threads_started":1}
I20260806 12:29:19.091408 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.085290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.6.152.131:34491)
0806 12:29:19.086809 (+ 1519us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.086820 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.086842 (+ 22us) server_negotiation.cc:407] Connection header received
0806 12:29:19.086914 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.086919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.087090 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.087292 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:19.087521 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.088551 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.090026 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.090675 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.090804 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.090947 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.091077 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.091203 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1387,"thread_start_us":205,"threads_started":1}
I20260806 12:29:19.091835 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.084851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.6.152.130:38753)
0806 12:29:19.085338 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.085349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.085374 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:29:19.085660 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.085666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.085857 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.086193 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:19.086602 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.087809 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.089597 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.090455 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.091253 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.091435 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.091491 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.091607 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":170,"threads_started":1}
I20260806 12:29:19.094738 14312 tablet_service.cc:1467] Processing CreateTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 (DEFAULT_TABLE table=test-table [id=e241bd72facb49318bb65df7c852002e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:19.094610 14452 tablet_service.cc:1467] Processing CreateTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 (DEFAULT_TABLE table=test-table [id=e241bd72facb49318bb65df7c852002e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:19.094574 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 (DEFAULT_TABLE table=test-table [id=e241bd72facb49318bb65df7c852002e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:19.095646 14312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:19.095690 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:19.095690 14452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:19.104496 14558 tablet_bootstrap.cc:492] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Bootstrap starting.
I20260806 12:29:19.105103 14560 tablet_bootstrap.cc:492] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Bootstrap starting.
I20260806 12:29:19.105747 14559 tablet_bootstrap.cc:492] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Bootstrap starting.
I20260806 12:29:19.107401 14558 tablet_bootstrap.cc:654] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:19.107890 14560 tablet_bootstrap.cc:654] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:19.108168 14559 tablet_bootstrap.cc:654] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:19.108521 14558 log.cc:824] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:19.109179 14560 log.cc:824] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:19.109558 14559 log.cc:824] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:19.110930 14558 tablet_bootstrap.cc:492] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: No bootstrap required, opened a new log
I20260806 12:29:19.111184 14558 ts_tablet_manager.cc:1397] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 12:29:19.111446 14559 tablet_bootstrap.cc:492] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: No bootstrap required, opened a new log
I20260806 12:29:19.111624 14559 ts_tablet_manager.cc:1397] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 12:29:19.115962 14558 raft_consensus.cc:348] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.116283 14558 raft_consensus.cc:374] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:19.116341 14558 raft_consensus.cc:729] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c6a1b3ad5643d29e070caba1f17fe0, State: Initialized, Role: FOLLOWER
I20260806 12:29:19.116569 14559 raft_consensus.cc:348] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.116822 14559 raft_consensus.cc:374] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:19.116879 14559 raft_consensus.cc:729] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d212ad55ae604ab68626673c7c547db2, State: Initialized, Role: FOLLOWER
I20260806 12:29:19.116808 14558 consensus_queue.cc:260] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.117300 14559 consensus_queue.cc:260] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.117578 14560 tablet_bootstrap.cc:492] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: No bootstrap required, opened a new log
I20260806 12:29:19.117779 14560 ts_tablet_manager.cc:1397] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260806 12:29:19.118258 14558 ts_tablet_manager.cc:1428] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 12:29:19.118407 14559 ts_tablet_manager.cc:1428] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 12:29:19.118680 14240 heartbeater.cc:503] Master 127.6.152.190:35853 was elected leader, sending a full tablet report...
I20260806 12:29:19.119211 14381 heartbeater.cc:503] Master 127.6.152.190:35853 was elected leader, sending a full tablet report...
I20260806 12:29:19.123854 14560 raft_consensus.cc:348] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.124119 14560 raft_consensus.cc:374] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:19.124182 14560 raft_consensus.cc:729] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b158c765e480291319a80e639c1d3, State: Initialized, Role: FOLLOWER
I20260806 12:29:19.124580 14560 consensus_queue.cc:260] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.125759 14560 ts_tablet_manager.cc:1428] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
I20260806 12:29:19.126200 14523 heartbeater.cc:503] Master 127.6.152.190:35853 was elected leader, sending a full tablet report...
I20260806 12:29:19.133602 14566 raft_consensus.cc:482] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:19.133752 14566 raft_consensus.cc:504] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.134931 14566 leader_election.cc:288] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471), 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491)
I20260806 12:29:19.139657 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.135460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:52101 (local address 127.6.152.131:34491)
0806 12:29:19.135670 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.135674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.135688 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:19.135922 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.135925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.135968 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.136145 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:19.136603 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.137070 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.138259 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.138970 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.139092 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.139168 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.139482 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.139548 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 12:29:19.139726 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.135471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:34491 (local address 127.6.152.130:52101)
0806 12:29:19.135579 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.135595 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.135663 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.136154 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.136158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.136175 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:19.136477 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.136487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.137188 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.137194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.138131 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.138141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.139094 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.139097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.139295 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.139343 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.139453 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.139537 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":18}
I20260806 12:29:19.140378 14474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfc0dedccb54dc5aa8ce7e429b93ba4" candidate_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b158c765e480291319a80e639c1d3" is_pre_election: true
I20260806 12:29:19.140666 14474 raft_consensus.cc:2454] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c6a1b3ad5643d29e070caba1f17fe0 in term 0.
I20260806 12:29:19.141017 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.135105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39471 (local address 127.6.152.130:47389)
0806 12:29:19.135237 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:19.135253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:19.135333 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:19.135772 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:19.135774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:19.135813 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:19.135948 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.135955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.138285 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.138287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.139621 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:19.139631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.140525 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.140532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:19.140693 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.140713 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 12:29:19.140776 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 12:29:19.140865 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":31}
I20260806 12:29:19.141017 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.135205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:47389 (local address 127.6.152.129:39471)
0806 12:29:19.135331 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.135337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.135356 (+ 19us) server_negotiation.cc:407] Connection header received
0806 12:29:19.135496 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.135502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.135551 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.135651 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:19.136426 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.137051 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.139771 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.140381 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.140510 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.140561 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.140828 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.140906 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:29:19.141453 14264 leader_election.cc:302] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 779b158c765e480291319a80e639c1d3, b0c6a1b3ad5643d29e070caba1f17fe0; no voters:
I20260806 12:29:19.141666 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfc0dedccb54dc5aa8ce7e429b93ba4" candidate_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212ad55ae604ab68626673c7c547db2" is_pre_election: true
I20260806 12:29:19.141710 14566 raft_consensus.cc:2781] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:19.141785 14566 raft_consensus.cc:482] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:19.141829 14566 raft_consensus.cc:3037] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:19.141974 14190 raft_consensus.cc:2454] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c6a1b3ad5643d29e070caba1f17fe0 in term 0.
I20260806 12:29:19.143455 14566 raft_consensus.cc:504] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.143755 14566 leader_election.cc:288] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [CANDIDATE]: Term 1 election: Requested vote from peers d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471), 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491)
I20260806 12:29:19.144228 14474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfc0dedccb54dc5aa8ce7e429b93ba4" candidate_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b158c765e480291319a80e639c1d3"
I20260806 12:29:19.144361 14474 raft_consensus.cc:3037] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:19.144352 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cfc0dedccb54dc5aa8ce7e429b93ba4" candidate_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212ad55ae604ab68626673c7c547db2"
I20260806 12:29:19.144488 14190 raft_consensus.cc:3037] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:19.145732 14190 raft_consensus.cc:2454] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c6a1b3ad5643d29e070caba1f17fe0 in term 1.
I20260806 12:29:19.146052 14264 leader_election.cc:302] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0c6a1b3ad5643d29e070caba1f17fe0, d212ad55ae604ab68626673c7c547db2; no voters:
I20260806 12:29:19.146271 14566 raft_consensus.cc:2781] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:19.146308 14474 raft_consensus.cc:2454] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c6a1b3ad5643d29e070caba1f17fe0 in term 1.
I20260806 12:29:19.146592 14566 raft_consensus.cc:686] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 LEADER]: Becoming Leader. State: Replica: b0c6a1b3ad5643d29e070caba1f17fe0, State: Running, Role: LEADER
I20260806 12:29:19.146910 14566 consensus_queue.cc:237] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:19.150298 13233 catalog_manager.cc:5495] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:19.174536 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.170894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.6.152.130:38753)
0806 12:29:19.171021 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.171025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.171042 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:29:19.171238 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.171240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.171303 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.171417 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:19.171747 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.172241 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.173106 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.173324 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.173434 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.173786 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:19.174149 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:19.174275 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.174340 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.174394 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:29:19.180892 14474 raft_consensus.cc:1261] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Refusing update from remote peer b0c6a1b3ad5643d29e070caba1f17fe0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:19.180899 14190 raft_consensus.cc:1261] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Refusing update from remote peer b0c6a1b3ad5643d29e070caba1f17fe0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:19.181599 14566 consensus_queue.cc:1035] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:19.181829 14569 consensus_queue.cc:1035] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:19.200788 13232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:19.200974 13232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:19.206653 13232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7f89a154cc04ef48cc7d43d56d5f845: Sending DeleteTablet for 3 replicas of tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4
I20260806 12:29:19.207387 14312 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:19 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:39056
I20260806 12:29:19.207439 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:19 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38856
I20260806 12:29:19.207572 14168 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:19 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38856
I20260806 12:29:19.208253 14452 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cfc0dedccb54dc5aa8ce7e429b93ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:19 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260806 12:29:19.208921 14582 tablet_replica.cc:330] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: stopping tablet replica
I20260806 12:29:19.209245 14582 raft_consensus.cc:2229] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:19.209515 14582 raft_consensus.cc:2258] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:19.210140 14585 tablet_replica.cc:330] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: stopping tablet replica
I20260806 12:29:19.210165 14583 tablet_replica.cc:330] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: stopping tablet replica
I20260806 12:29:19.210446 14583 raft_consensus.cc:2229] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:19.210482 14585 raft_consensus.cc:2229] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:19.210496 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13199
I20260806 12:29:19.212013 14583 raft_consensus.cc:2258] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:19.212118 14582 ts_tablet_manager.cc:1905] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:19.212206 14585 raft_consensus.cc:2258] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:19.213464 14583 ts_tablet_manager.cc:1905] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:19.213625 14585 ts_tablet_manager.cc:1905] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:19.215525 14582 ts_tablet_manager.cc:1918] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:19.215646 14582 log.cc:1196] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/8cfc0dedccb54dc5aa8ce7e429b93ba4
I20260806 12:29:19.215996 14582 ts_tablet_manager.cc:1939] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P d212ad55ae604ab68626673c7c547db2: Deleting consensus metadata
I20260806 12:29:19.216784 14585 ts_tablet_manager.cc:1918] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:19.216872 14585 log.cc:1196] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/8cfc0dedccb54dc5aa8ce7e429b93ba4
I20260806 12:29:19.217187 14585 ts_tablet_manager.cc:1939] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P 779b158c765e480291319a80e639c1d3: Deleting consensus metadata
I20260806 12:29:19.217191 14583 ts_tablet_manager.cc:1918] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:19.217281 14583 log.cc:1196] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/8cfc0dedccb54dc5aa8ce7e429b93ba4
I20260806 12:29:19.217628 14583 ts_tablet_manager.cc:1939] T 8cfc0dedccb54dc5aa8ce7e429b93ba4 P b0c6a1b3ad5643d29e070caba1f17fe0: Deleting consensus metadata
W20260806 12:29:19.220731 14405 connection.cc:535] server connection from 127.0.0.1:38244 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 12:29:19.220750 14265 connection.cc:535] server connection from 127.0.0.1:39056 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 12:29:19.221118 14122 connection.cc:535] server connection from 127.0.0.1:38856 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 12:29:19.231077 14533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.209836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.6.152.189:44319)
0806 12:29:19.209974 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.209980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.220617 (+ 10637us) server_negotiation.cc:407] Connection header received
0806 12:29:19.220836 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.220839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.220895 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.220951 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.222156 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.222980 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.224239 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.224618 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.224695 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.226934 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.226938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.226941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.226967 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.228788 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.229305 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.229308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.229310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.229406 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.229721 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.229725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.229726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.229914 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.230455 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.230808 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.230922 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:29:19.236195 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.225001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43546 (local address 127.6.152.189:44319)
0806 12:29:19.225126 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.225130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.225423 (+ 293us) server_negotiation.cc:407] Connection header received
0806 12:29:19.225617 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.225623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.225675 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.225724 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.226955 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.227824 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.229091 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.229432 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.229504 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.231829 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.231833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.231836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.231862 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.233941 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.234664 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.234671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.234674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.234769 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.235212 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.235215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.235217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.235428 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.235533 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.235910 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.236028 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 12:29:19.237396 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.224771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.6.152.188:35583)
0806 12:29:19.224900 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.224904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.225446 (+ 542us) server_negotiation.cc:407] Connection header received
0806 12:29:19.225695 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.225701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.225751 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.225802 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.227305 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.228023 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.229184 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.229512 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.229603 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.231331 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.231334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.231337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.231360 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.235589 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.236031 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.236034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.236036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.236114 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.236432 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.236437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.236438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.236659 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.236764 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.237157 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.237259 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:19.239703 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:19.212837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.6.152.188:35583)
0806 12:29:19.212959 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:19.212963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:19.231575 (+ 18612us) server_negotiation.cc:407] Connection header received
0806 12:29:19.231733 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:19.231737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:19.231784 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:19.231835 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:19.232547 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.233293 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.234395 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:19.234728 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:19.234797 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:19.236307 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:19.236310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:19.236313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:19.236347 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:19.237947 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.238398 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.238402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.238403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.238477 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:19.238758 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:19.238761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:19.238762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:19.238907 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:19.239019 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:19.239406 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 12:29:19.239542 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:29:20.194823 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.194351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:55793)
0806 12:29:20.194613 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.194697 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":97,"threads_started":1}
W20260806 12:29:20.195199 14523 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:35853 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:20.200712 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.199753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:58813)
0806 12:29:20.200491 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.200606 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":151,"threads_started":1}
W20260806 12:29:20.200999 14381 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:35853 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:20.208091 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.207716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:57943)
0806 12:29:20.207938 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.208012 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
W20260806 12:29:20.208400 14240 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:35853 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:20.706900 14598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b7f89a154cc04ef48cc7d43d56d5f845)
I20260806 12:29:20.706889 14597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b7f89a154cc04ef48cc7d43d56d5f845)
I20260806 12:29:20.707129 14598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:20.707171 14597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:20.708165 14597 leader_election.cc:288] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853), 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189:44319)
I20260806 12:29:20.708775 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.708320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58536)
0806 12:29:20.708586 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.708673 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":60,"threads_started":1}
I20260806 12:29:20.708775 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.708300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58538)
0806 12:29:20.708617 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.708676 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":60,"threads_started":1}
I20260806 12:29:20.708811 14598 leader_election.cc:288] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853), 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188:35583)
W20260806 12:29:20.708961 13849 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
W20260806 12:29:20.709383 14038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:20.710314 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.709934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58564)
0806 12:29:20.710174 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.710229 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260806 12:29:20.710476 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.710241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58570)
0806 12:29:20.710365 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.710410 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 12:29:20.710496 13849 leader_election.cc:334] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
W20260806 12:29:20.710677 14038 leader_election.cc:334] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:20.712736 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.708312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:35583 (local address 127.0.0.1:36974)
0806 12:29:20.709155 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.709168 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:20.709268 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:20.709713 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:20.709718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:20.709737 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:20.709830 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:20.709837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.710840 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.710843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:20.711668 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:20.711678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.712317 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.712321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:20.712420 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:20.712452 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 12:29:20.712523 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:29:20.712597 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260806 12:29:20.712777 14602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.708674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.6.152.188:35583)
0806 12:29:20.709191 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:20.709197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:20.709216 (+ 19us) server_negotiation.cc:407] Connection header received
0806 12:29:20.709400 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:20.709403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:20.709452 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:20.709527 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:20.709966 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.710690 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.711786 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.712208 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.712290 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:20.712338 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:20.712561 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 12:29:20.712644 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":68,"threads_started":1}
I20260806 12:29:20.712913 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.708708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.6.152.189:44319)
0806 12:29:20.709069 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:20.709075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:20.709299 (+ 224us) server_negotiation.cc:407] Connection header received
0806 12:29:20.709401 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:20.709414 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:20.709469 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:20.709551 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:20.709935 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.710690 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.711816 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.712363 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.712463 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:20.712531 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:20.712698 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 12:29:20.712771 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":65,"threads_started":1}
I20260806 12:29:20.712955 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.708908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:44319 (local address 127.0.0.1:59876)
0806 12:29:20.709068 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.709083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:20.709206 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:20.709685 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:20.709688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:20.709709 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:20.709811 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:20.709819 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.710837 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.710845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:20.711667 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:20.711676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:20.712473 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:20.712475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:20.712576 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:20.712595 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 12:29:20.712671 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:29:20.712727 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260806 12:29:20.713382 14067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54894143a9974e90b1f1aa1a145d4efd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" is_pre_election: true
I20260806 12:29:20.713382 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54894143a9974e90b1f1aa1a145d4efd" is_pre_election: true
I20260806 12:29:20.713585 14067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54894143a9974e90b1f1aa1a145d4efd in term 1.
I20260806 12:29:20.713585 13872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e65e23e4cec4adcb3b40b1ffefaf192 in term 1.
I20260806 12:29:20.713956 13846 leader_election.cc:302] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e65e23e4cec4adcb3b40b1ffefaf192, 54894143a9974e90b1f1aa1a145d4efd; no voters: b7f89a154cc04ef48cc7d43d56d5f845
I20260806 12:29:20.713987 14035 leader_election.cc:302] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e65e23e4cec4adcb3b40b1ffefaf192, 54894143a9974e90b1f1aa1a145d4efd; no voters: b7f89a154cc04ef48cc7d43d56d5f845
I20260806 12:29:20.714201 14597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 12:29:20.714197 14598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 12:29:20.714263 14597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b7f89a154cc04ef48cc7d43d56d5f845)
I20260806 12:29:20.714263 14598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Starting leader election (detected failure of leader b7f89a154cc04ef48cc7d43d56d5f845)
I20260806 12:29:20.714308 14598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:29:20.714308 14597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:29:20.715585 14597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:20.715585 14598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:20.715950 14597 leader_election.cc:288] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 2 election: Requested vote from peers b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853), 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189:44319)
I20260806 12:29:20.715956 14598 leader_election.cc:288] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [CANDIDATE]: Term 2 election: Requested vote from peers b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853), 1e65e23e4cec4adcb3b40b1ffefaf192 (127.6.152.188:35583)
I20260806 12:29:20.716259 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54894143a9974e90b1f1aa1a145d4efd"
I20260806 12:29:20.716444 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.716068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58572)
0806 12:29:20.716329 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.716380 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 12:29:20.716429 14067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54894143a9974e90b1f1aa1a145d4efd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192"
I20260806 12:29:20.716512 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.716258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58578)
0806 12:29:20.716394 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.716460 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 12:29:20.716446 13872 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1e65e23e4cec4adcb3b40b1ffefaf192 in current term 2: Already voted for candidate 54894143a9974e90b1f1aa1a145d4efd in this term.
I20260806 12:29:20.716578 14067 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 54894143a9974e90b1f1aa1a145d4efd in current term 2: Already voted for candidate 1e65e23e4cec4adcb3b40b1ffefaf192 in this term.
I20260806 12:29:20.717110 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.716877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58588)
0806 12:29:20.717003 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.717065 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 12:29:20.717140 14604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:20.716948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58604)
0806 12:29:20.717030 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:20.717082 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 12:29:20.717240 13849 leader_election.cc:334] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
W20260806 12:29:20.717272 14038 leader_election.cc:334] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:20.717314 13849 leader_election.cc:302] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54894143a9974e90b1f1aa1a145d4efd; no voters: 1e65e23e4cec4adcb3b40b1ffefaf192, b7f89a154cc04ef48cc7d43d56d5f845
I20260806 12:29:20.717324 14038 leader_election.cc:302] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e65e23e4cec4adcb3b40b1ffefaf192; no voters: 54894143a9974e90b1f1aa1a145d4efd, b7f89a154cc04ef48cc7d43d56d5f845
I20260806 12:29:20.717453 14597 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 12:29:20.717618 14598 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 12:29:21.196265 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:21.195777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:45363)
0806 12:29:21.196085 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:21.196173 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":88,"threads_started":1}
I20260806 12:29:21.201942 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:21.201545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:59959)
0806 12:29:21.201800 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:21.201856 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260806 12:29:21.209249 14610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:21.208868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:52369)
0806 12:29:21.209110 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:21.209167 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260806 12:29:22.197516 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.197007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:43059)
0806 12:29:22.197306 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.197401 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
I20260806 12:29:22.202947 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.202574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:48053)
0806 12:29:22.202804 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.202860 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260806 12:29:22.210258 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.209913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:43569)
0806 12:29:22.210125 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.210175 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":65,"threads_started":1}
I20260806 12:29:22.266911 14614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:22.267050 14614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:22.267592 14614 leader_election.cc:288] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853), 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189:44319)
I20260806 12:29:22.268127 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54894143a9974e90b1f1aa1a145d4efd" is_pre_election: true
I20260806 12:29:22.268275 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.267825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58806)
0806 12:29:22.268098 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.268173 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":66,"threads_started":1}
I20260806 12:29:22.268294 13872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e65e23e4cec4adcb3b40b1ffefaf192 in term 2.
I20260806 12:29:22.268628 14035 leader_election.cc:302] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e65e23e4cec4adcb3b40b1ffefaf192, 54894143a9974e90b1f1aa1a145d4efd; no voters:
I20260806 12:29:22.268800 14614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 12:29:22.268883 14614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:22.268939 14614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:29:22.269269 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.269012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58816)
0806 12:29:22.269134 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.269188 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 12:29:22.269392 14038 leader_election.cc:334] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:22.270182 14614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:22.270520 14614 leader_election.cc:288] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 3 election: Requested vote from peers b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853), 54894143a9974e90b1f1aa1a145d4efd (127.6.152.189:44319)
I20260806 12:29:22.270840 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.270626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58830)
0806 12:29:22.270723 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.270778 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:29:22.270836 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54894143a9974e90b1f1aa1a145d4efd"
I20260806 12:29:22.270983 13872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:29:22.271471 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.271265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58838)
0806 12:29:22.271364 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.271423 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 12:29:22.271569 14038 leader_election.cc:334] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
I20260806 12:29:22.272090 13872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e65e23e4cec4adcb3b40b1ffefaf192 in term 3.
I20260806 12:29:22.272392 14035 leader_election.cc:302] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e65e23e4cec4adcb3b40b1ffefaf192, 54894143a9974e90b1f1aa1a145d4efd; no voters: b7f89a154cc04ef48cc7d43d56d5f845
I20260806 12:29:22.272575 14614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 12:29:22.272821 14614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [term 3 LEADER]: Becoming Leader. State: Replica: 1e65e23e4cec4adcb3b40b1ffefaf192, State: Running, Role: LEADER
I20260806 12:29:22.273104 14614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } }
I20260806 12:29:22.274439 14617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e65e23e4cec4adcb3b40b1ffefaf192. Latest consensus state: current_term: 3 leader_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:22.274561 14617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:22.274822 14619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:22.277348 14619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:22.277741 14619 catalog_manager.cc:1259] Loaded cluster ID: d2e58b97f0624335b35407b3613d2fc6
I20260806 12:29:22.277786 14619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:22.278465 14619 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:22.278867 14619 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Loaded TSK: 0
I20260806 12:29:22.279582 14619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:22.286748 14054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:22.289876 14054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:22.292788 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.292561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58860)
0806 12:29:22.292668 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.292733 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 12:29:22.292848 13872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [term 3 FOLLOWER]: Refusing update from remote peer 1e65e23e4cec4adcb3b40b1ffefaf192: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 12:29:22.293438 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.293224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.0.0.1:58868)
0806 12:29:22.293338 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.293392 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 12:29:22.293596 14614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 12:29:22.293798 14038 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 -> Peer b7f89a154cc04ef48cc7d43d56d5f845 (127.6.152.190:35853): Couldn't send request to peer b7f89a154cc04ef48cc7d43d56d5f845. Status: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 12:29:22.294960 14621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e65e23e4cec4adcb3b40b1ffefaf192. Latest consensus state: current_term: 3 leader_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:22.295083 14621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:22.296108 14617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:22.296221 14617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:22.297267 14621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7f89a154cc04ef48cc7d43d56d5f845" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35853 } } peers { permanent_uuid: "54894143a9974e90b1f1aa1a145d4efd" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 44319 } attrs { promote: false } } peers { permanent_uuid: "1e65e23e4cec4adcb3b40b1ffefaf192" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 35583 } attrs { promote: false } } }
I20260806 12:29:22.297372 14621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54894143a9974e90b1f1aa1a145d4efd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:29:22.306717 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.301400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:38753 (local address 127.0.0.1:59944)
0806 12:29:22.301485 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.301498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:22.301817 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:22.302344 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:22.302350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:22.302374 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:22.302486 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.302493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.303693 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.303697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.305017 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.305026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.306350 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.306353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.306450 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.306473 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:29:22.306542 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 12:29:22.306603 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":22}
I20260806 12:29:22.307232 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.301516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.6.152.130:38753)
0806 12:29:22.301774 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:22.301792 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:22.301810 (+ 18us) server_negotiation.cc:407] Connection header received
0806 12:29:22.301943 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:22.301949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:22.302012 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:22.302622 (+ 610us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:22.302742 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.303559 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.305202 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.305770 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.305908 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.305969 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:22.306941 (+ 972us) server_negotiation.cc:299] Negotiation successful
0806 12:29:22.307020 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":67,"threads_started":1}
I20260806 12:29:22.307462 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.302750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.6.152.129:39471)
0806 12:29:22.303045 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:22.303050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:22.303062 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:29:22.303110 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:22.303112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:22.303183 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:22.303943 (+ 760us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:22.304082 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.304710 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.306269 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.306696 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.306925 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.306988 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:22.307255 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 12:29:22.307330 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":98,"threads_started":1}
I20260806 12:29:22.307611 14628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.302279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:34491 (local address 127.0.0.1:42850)
0806 12:29:22.302523 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.302537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:22.302725 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:22.303452 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:22.303457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:22.303820 (+ 363us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:22.303971 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.303979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.304991 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.304995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.306195 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.306205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.307148 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.307152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.307233 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.307285 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 12:29:22.307348 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 12:29:22.307421 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":76,"spinlock_wait_cycles":730496,"thread_start_us":96,"threads_started":1}
I20260806 12:29:22.307746 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.301905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39471 (local address 127.0.0.1:45230)
0806 12:29:22.302095 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.302106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:22.302168 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:22.303397 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:22.303402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:22.303685 (+ 283us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:22.303823 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.303831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.304818 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.304823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.306144 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.306153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.306826 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.306830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.307063 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.307092 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 12:29:22.307229 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 12:29:22.307433 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":62,"thread_start_us":81,"threads_started":1}
I20260806 12:29:22.307866 14312 tablet_service.cc:1467] Processing CreateTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 (DEFAULT_TABLE table=test-table [id=83b8218d0b994551be3cadd830e3aea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:22.308162 14312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2361c85c7a24ccba287e1ce55d0b143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:22.308212 14168 tablet_service.cc:1467] Processing CreateTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 (DEFAULT_TABLE table=test-table [id=83b8218d0b994551be3cadd830e3aea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:22.308542 14168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2361c85c7a24ccba287e1ce55d0b143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:22.310672 14631 tablet_bootstrap.cc:492] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Bootstrap starting.
I20260806 12:29:22.311003 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.302379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.6.152.131:34491)
0806 12:29:22.302959 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:22.302969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:22.302984 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:29:22.303056 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:22.303061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:22.303113 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:22.303581 (+ 468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:22.304111 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.304883 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.306321 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.306990 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.310688 (+ 3698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.310752 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:22.310785 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 12:29:22.310851 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":72,"threads_started":1}
I20260806 12:29:22.311228 14632 tablet_bootstrap.cc:492] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: Bootstrap starting.
I20260806 12:29:22.311630 14452 tablet_service.cc:1467] Processing CreateTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 (DEFAULT_TABLE table=test-table [id=83b8218d0b994551be3cadd830e3aea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:22.311895 14452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2361c85c7a24ccba287e1ce55d0b143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:22.312103 14632 tablet_bootstrap.cc:654] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:22.313495 14632 tablet_bootstrap.cc:492] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: No bootstrap required, opened a new log
I20260806 12:29:22.313611 14632 ts_tablet_manager.cc:1397] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:22.314054 14632 raft_consensus.cc:348] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.314211 14632 raft_consensus.cc:374] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:22.314282 14632 raft_consensus.cc:729] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d212ad55ae604ab68626673c7c547db2, State: Initialized, Role: FOLLOWER
I20260806 12:29:22.314453 14632 consensus_queue.cc:260] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.314571 14634 tablet_bootstrap.cc:492] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Bootstrap starting.
I20260806 12:29:22.314965 14632 ts_tablet_manager.cc:1428] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 12:29:22.314987 14631 tablet_bootstrap.cc:654] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:22.315510 14634 tablet_bootstrap.cc:654] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:22.316216 14631 tablet_bootstrap.cc:492] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: No bootstrap required, opened a new log
I20260806 12:29:22.316316 14631 ts_tablet_manager.cc:1397] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260806 12:29:22.316664 14631 raft_consensus.cc:348] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.316742 14631 raft_consensus.cc:374] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:22.316762 14631 raft_consensus.cc:729] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c6a1b3ad5643d29e070caba1f17fe0, State: Initialized, Role: FOLLOWER
I20260806 12:29:22.316787 14634 tablet_bootstrap.cc:492] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: No bootstrap required, opened a new log
I20260806 12:29:22.316859 14634 ts_tablet_manager.cc:1397] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:22.316885 14631 consensus_queue.cc:260] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.317250 14631 ts_tablet_manager.cc:1428] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:29:22.317269 14634 raft_consensus.cc:348] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.317363 14634 raft_consensus.cc:374] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:22.317404 14634 raft_consensus.cc:729] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b158c765e480291319a80e639c1d3, State: Initialized, Role: FOLLOWER
I20260806 12:29:22.317513 14634 consensus_queue.cc:260] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.317734 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.316027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:56405)
0806 12:29:22.317597 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.317657 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1496}
I20260806 12:29:22.317849 14634 ts_tablet_manager.cc:1428] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:22.318143 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.317837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:46407)
0806 12:29:22.318042 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.318094 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260806 12:29:22.318785 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.318506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:47127)
0806 12:29:22.318666 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.318719 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 12:29:22.319500 14239 heartbeater.cc:503] Master 127.6.152.188:35583 was elected leader, sending a full tablet report...
I20260806 12:29:22.319772 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.319500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:53131)
0806 12:29:22.319651 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.319703 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 12:29:22.319902 14380 heartbeater.cc:503] Master 127.6.152.188:35583 was elected leader, sending a full tablet report...
I20260806 12:29:22.320322 14522 heartbeater.cc:503] Master 127.6.152.188:35583 was elected leader, sending a full tablet report...
I20260806 12:29:22.321590 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.320219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:59927)
0806 12:29:22.321468 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.321526 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1176}
I20260806 12:29:22.322149 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.321913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:54375)
0806 12:29:22.322028 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.322077 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 12:29:22.336150 14383 tablet.cc:2369] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:29:22.368523 14242 tablet.cc:2369] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:29:22.400347 14640 raft_consensus.cc:482] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:22.400511 14640 raft_consensus.cc:504] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.402000 14640 leader_election.cc:288] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753), d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471)
I20260806 12:29:22.405772 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.402201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:38753 (local address 127.6.152.131:44789)
0806 12:29:22.402401 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.402437 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:22.402548 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:22.402805 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:22.402808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:22.402825 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:22.402938 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.402949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.403811 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.403818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.404721 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.404730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.405373 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.405377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.405465 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.405493 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 12:29:22.405571 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:29:22.405663 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 12:29:22.405822 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.402290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:44789 (local address 127.6.152.130:38753)
0806 12:29:22.402438 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:22.402443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:22.402529 (+ 86us) server_negotiation.cc:407] Connection header received
0806 12:29:22.402650 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:22.402655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:22.402706 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:22.402792 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:22.403065 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.403681 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.404846 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.405250 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.405380 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.405458 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:22.405610 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 12:29:22.405690 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 12:29:22.406314 14332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2361c85c7a24ccba287e1ce55d0b143" candidate_uuid: "779b158c765e480291319a80e639c1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" is_pre_election: true
I20260806 12:29:22.406446 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.402229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39471 (local address 127.6.152.131:52211)
0806 12:29:22.402603 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.402614 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:22.402741 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:22.403010 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:22.403013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:22.403029 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:22.403149 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.403159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.404017 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.404023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.404909 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:22.404918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.406055 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.406059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:22.406132 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.406150 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:29:22.406238 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 12:29:22.406299 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":137,"threads_started":1}
I20260806 12:29:22.406461 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.402351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:52211 (local address 127.6.152.129:39471)
0806 12:29:22.402476 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:22.402484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:22.402700 (+ 216us) server_negotiation.cc:407] Connection header received
0806 12:29:22.402854 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:22.402857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:22.402909 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:22.403000 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:22.403288 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.403893 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.405051 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.405471 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.405653 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.405697 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:22.406260 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 12:29:22.406358 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:29:22.406590 14332 raft_consensus.cc:2454] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 779b158c765e480291319a80e639c1d3 in term 0.
I20260806 12:29:22.406905 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2361c85c7a24ccba287e1ce55d0b143" candidate_uuid: "779b158c765e480291319a80e639c1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212ad55ae604ab68626673c7c547db2" is_pre_election: true
I20260806 12:29:22.407055 14190 raft_consensus.cc:2454] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 779b158c765e480291319a80e639c1d3 in term 0.
I20260806 12:29:22.407451 14405 leader_election.cc:302] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 779b158c765e480291319a80e639c1d3, d212ad55ae604ab68626673c7c547db2; no voters:
I20260806 12:29:22.407683 14640 raft_consensus.cc:2781] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:22.407743 14640 raft_consensus.cc:482] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:22.407771 14640 raft_consensus.cc:3037] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:22.408769 14640 raft_consensus.cc:504] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.409144 14640 leader_election.cc:288] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [CANDIDATE]: Term 1 election: Requested vote from peers b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753), d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471)
I20260806 12:29:22.409554 14332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2361c85c7a24ccba287e1ce55d0b143" candidate_uuid: "779b158c765e480291319a80e639c1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0"
I20260806 12:29:22.409639 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2361c85c7a24ccba287e1ce55d0b143" candidate_uuid: "779b158c765e480291319a80e639c1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212ad55ae604ab68626673c7c547db2"
I20260806 12:29:22.409704 14332 raft_consensus.cc:3037] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:22.409802 14190 raft_consensus.cc:3037] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:22.410737 14332 raft_consensus.cc:2454] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 779b158c765e480291319a80e639c1d3 in term 1.
I20260806 12:29:22.410797 14190 raft_consensus.cc:2454] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 779b158c765e480291319a80e639c1d3 in term 1.
I20260806 12:29:22.411070 14408 leader_election.cc:302] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 779b158c765e480291319a80e639c1d3, b0c6a1b3ad5643d29e070caba1f17fe0, d212ad55ae604ab68626673c7c547db2; no voters:
I20260806 12:29:22.411343 14640 raft_consensus.cc:2781] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:22.411536 14640 raft_consensus.cc:686] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 1 LEADER]: Becoming Leader. State: Replica: 779b158c765e480291319a80e639c1d3, State: Running, Role: LEADER
I20260806 12:29:22.411784 14640 consensus_queue.cc:237] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } }
I20260806 12:29:22.412225 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.411988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:41947)
0806 12:29:22.412105 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.412161 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 12:29:22.414700 14055 catalog_manager.cc:5495] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 779b158c765e480291319a80e639c1d3 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "779b158c765e480291319a80e639c1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "779b158c765e480291319a80e639c1d3" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34491 } health_report { overall_health: HEALTHY } } }
I20260806 12:29:22.433087 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.429569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.6.152.131:34491)
0806 12:29:22.429686 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:22.429691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:22.429730 (+ 39us) server_negotiation.cc:407] Connection header received
0806 12:29:22.429916 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:22.429919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:22.429970 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:22.430084 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:22.430312 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.430794 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.431706 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:22.431917 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:22.432025 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:22.432288 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:22.432692 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:22.432816 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:22.432880 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 12:29:22.432931 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:29:22.435705 14190 raft_consensus.cc:1261] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Refusing update from remote peer 779b158c765e480291319a80e639c1d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:22.435950 14332 raft_consensus.cc:1261] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Refusing update from remote peer 779b158c765e480291319a80e639c1d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:22.436232 14640 consensus_queue.cc:1035] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d212ad55ae604ab68626673c7c547db2" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:22.436551 14642 consensus_queue.cc:1035] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0c6a1b3ad5643d29e070caba1f17fe0" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 38753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:22.438934 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.438667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:54649)
0806 12:29:22.438811 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.438868 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 12:29:22.438962 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.438666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:57071)
0806 12:29:22.438820 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.438876 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 12:29:22.440464 14645 mvcc.cc:205] Tried to move back new op lower bound from 7315535308531535872 to 7315535308441481216. Current Snapshot: MvccSnapshot[applied={T|T < 7315535308531535872}]
I20260806 12:29:22.441166 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.440892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:47083)
0806 12:29:22.441031 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.441085 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 12:29:22.442071 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.441728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.130:57479)
0806 12:29:22.441963 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.442013 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260806 12:29:22.443943 14648 mvcc.cc:205] Tried to move back new op lower bound from 7315535308531535872 to 7315535308441481216. Current Snapshot: MvccSnapshot[applied={T|T < 7315535308531535872}]
I20260806 12:29:22.445891 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.445592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.129:41001)
0806 12:29:22.445773 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.445825 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 12:29:22.448930 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.444643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35853 (local address 127.6.152.131:42879)
0806 12:29:22.448807 (+ 4164us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.448865 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4077}
I20260806 12:29:22.449468 14055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:22.449640 14055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:22.454219 14055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e65e23e4cec4adcb3b40b1ffefaf192: Sending DeleteTablet for 3 replicas of tablet f2361c85c7a24ccba287e1ce55d0b143
I20260806 12:29:22.454742 14312 tablet_service.cc:1514] Processing DeleteTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:22 UTC) from {username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:59944
I20260806 12:29:22.455120 14168 tablet_service.cc:1514] Processing DeleteTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:22 UTC) from {username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:45230
I20260806 12:29:22.455314 14452 tablet_service.cc:1514] Processing DeleteTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:22 UTC) from {username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:42850
I20260806 12:29:22.455581 14655 tablet_replica.cc:330] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2: stopping tablet replica
I20260806 12:29:22.455597 14451 tablet_service.cc:1514] Processing DeleteTablet for tablet f2361c85c7a24ccba287e1ce55d0b143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:22 UTC) from {username='oryx', principal='oryx/127.6.152.188@KRBTEST.COM'} at 127.0.0.1:42850
I20260806 12:29:22.455746 14655 raft_consensus.cc:2229] T f2361c85c7a24ccba287e1ce55d0b143 P d212ad55ae604ab68626673c7c547db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:22.455825 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 14013
I20260806 12:29:22.457142 14656 tablet_replica.cc:330] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: stopping tablet replica
I20260806 12:29:22.457278 14656 raft_consensus.cc:2229] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:22.457712 14656 raft_consensus.cc:2258] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:22.458562 14656 ts_tablet_manager.cc:1905] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:22.458741 14654 tablet_replica.cc:330] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: stopping tablet replica
I20260806 12:29:22.458913 14654 raft_consensus.cc:2229] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:22.459111 14654 raft_consensus.cc:2258] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:22.459810 14654 ts_tablet_manager.cc:1905] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:22.461475 14656 ts_tablet_manager.cc:1918] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:22.461619 14656 log.cc:1196] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/f2361c85c7a24ccba287e1ce55d0b143
I20260806 12:29:22.461967 14656 ts_tablet_manager.cc:1939] T f2361c85c7a24ccba287e1ce55d0b143 P 779b158c765e480291319a80e639c1d3: Deleting consensus metadata
I20260806 12:29:22.462739 14654 ts_tablet_manager.cc:1918] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:22.462711 14035 catalog_manager.cc:4841] TS 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491): tablet f2361c85c7a24ccba287e1ce55d0b143 (table test-table [id=83b8218d0b994551be3cadd830e3aea6]) successfully deleted
I20260806 12:29:22.462893 14654 log.cc:1196] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/f2361c85c7a24ccba287e1ce55d0b143
I20260806 12:29:22.463336 14654 ts_tablet_manager.cc:1939] T f2361c85c7a24ccba287e1ce55d0b143 P b0c6a1b3ad5643d29e070caba1f17fe0: Deleting consensus metadata
W20260806 12:29:22.463414 14035 catalog_manager.cc:4808] TS 779b158c765e480291319a80e639c1d3 (127.6.152.131:34491): delete failed for tablet f2361c85c7a24ccba287e1ce55d0b143 because the tablet was not found. No further retry: Not found: Tablet not found: f2361c85c7a24ccba287e1ce55d0b143
I20260806 12:29:22.464020 14038 catalog_manager.cc:4841] TS b0c6a1b3ad5643d29e070caba1f17fe0 (127.6.152.130:38753): tablet f2361c85c7a24ccba287e1ce55d0b143 (table test-table [id=83b8218d0b994551be3cadd830e3aea6]) successfully deleted
I20260806 12:29:22.465927 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:22.465682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39471 (local address 127.0.0.1:45236)
0806 12:29:22.465812 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:22.465866 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 12:29:22.466157 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 14247
W20260806 12:29:22.468369 14035 catalog_manager.cc:4584] TS d212ad55ae604ab68626673c7c547db2 (127.6.152.129:39471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2361c85c7a24ccba287e1ce55d0b143: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39471: connect: Connection refused (error 111)
I20260806 12:29:22.474298 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 14387
I20260806 12:29:22.480990 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13718
I20260806 12:29:22.487713 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 13826
2026-08-06T12:29:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6888 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:22 dist-test-slave-1xrh krb5kdc[14659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:22 dist-test-slave-1xrh krb5kdc[14659](info): set up 2 sockets
Aug 06 12:29:22 dist-test-slave-1xrh krb5kdc[14659](info): commencing operation
krb5kdc: starting...
W20260806 12:29:23.562984 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:23 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab.
Entry for principal oryx/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab.
I20260806 12:29:23.686381 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45863
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab
--principal=oryx/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:45863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:23.797096 14672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:23.797405 14672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:23.797479 14672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:23.801512 14672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:23.801584 14672 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 12:29:23.801614 14672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:23.801656 14672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:23.801705 14672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:23.801743 14672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:23.807003 14672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:45863
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.190.keytab
--principal=oryx/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:23.808074 14672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:23.809094 14672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:23.817608 14680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:23.817787 14677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:23.817651 14678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:23.817905 14672 server_base.cc:1034] running on GCE node
I20260806 12:29:23.818481 14672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:23.819557 14672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:23.820716 14672 hybrid_clock.cc:648] HybridClock initialized: now 1786019363820694 us; error 34 us; skew 500 ppm
Aug 06 12:29:23 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019363, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:23.823462 14672 init.cc:377] Logged in from keytab as oryx/127.6.152.190@KRBTEST.COM (short username oryx)
I20260806 12:29:23.824312 14672 webserver.cc:460] Webserver started at http://127.6.152.190:39057/ using document root <none> and password file <none>
I20260806 12:29:23.824801 14672 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:23.824860 14672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:23.825024 14672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:23.826947 14672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "69fee10a67c8497d87a0dd9e9258c0bf"
format_stamp: "Formatted at 2026-08-06 12:29:23 on dist-test-slave-1xrh"
server_key: "1872bc62a25eabf8abdba33aea9b2e35"
server_key_iv: "03d4c8ef11e74f325dc2ecb09134ff65"
server_key_version: "encryptionkey@0"
I20260806 12:29:23.827521 14672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "69fee10a67c8497d87a0dd9e9258c0bf"
format_stamp: "Formatted at 2026-08-06 12:29:23 on dist-test-slave-1xrh"
server_key: "1872bc62a25eabf8abdba33aea9b2e35"
server_key_iv: "03d4c8ef11e74f325dc2ecb09134ff65"
server_key_version: "encryptionkey@0"
I20260806 12:29:23.830989 14672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:29:23.833320 14687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:23.834504 14672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:23.834604 14672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "69fee10a67c8497d87a0dd9e9258c0bf"
format_stamp: "Formatted at 2026-08-06 12:29:23 on dist-test-slave-1xrh"
server_key: "1872bc62a25eabf8abdba33aea9b2e35"
server_key_iv: "03d4c8ef11e74f325dc2ecb09134ff65"
server_key_version: "encryptionkey@0"
I20260806 12:29:23.834686 14672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:23.850852 14672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:23.853628 14672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:23.853793 14672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:23.861896 14672 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:45863
I20260806 12:29:23.861910 14755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:45863 every 8 connection(s)
I20260806 12:29:23.862828 14672 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:23.865563 14756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:23.870939 14756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Bootstrap starting.
I20260806 12:29:23.872539 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 14672
I20260806 12:29:23.872671 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:23.872968 6754 external_mini_cluster.cc:1428] Setting key 32589648887481d281f18910c0b1041f
I20260806 12:29:23.873553 14756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:23.874436 14756 log.cc:824] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:23.876513 14756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: No bootstrap required, opened a new log
I20260806 12:29:23.880695 14756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45863 } }
I20260806 12:29:23.881011 14756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:23.881084 14756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69fee10a67c8497d87a0dd9e9258c0bf, State: Initialized, Role: FOLLOWER
I20260806 12:29:23.881621 14756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45863 } }
I20260806 12:29:23.881779 14756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:23.881867 14756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:23.881964 14756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 0 FOLLOWER]: Advancing to term 1
Aug 06 12:29:23 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:23.882964 14756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45863 } }
I20260806 12:29:23.883332 14756 leader_election.cc:302] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69fee10a67c8497d87a0dd9e9258c0bf; no voters:
I20260806 12:29:23.883721 14756 leader_election.cc:288] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:23.883837 14761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:23.884089 14761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [term 1 LEADER]: Becoming Leader. State: Replica: 69fee10a67c8497d87a0dd9e9258c0bf, State: Running, Role: LEADER
I20260806 12:29:23.884425 14761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45863 } }
I20260806 12:29:23.885237 14756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:23.885560 14761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69fee10a67c8497d87a0dd9e9258c0bf. Latest consensus state: current_term: 1 leader_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45863 } } }
I20260806 12:29:23.885560 14762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fee10a67c8497d87a0dd9e9258c0bf" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45863 } } }
I20260806 12:29:23.885716 14761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:23.885721 14762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:23.886097 14765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:23.888785 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:23.874138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.6.152.190:45863)
0806 12:29:23.874651 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:23.874667 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:23.874701 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:29:23.875542 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:23.875561 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:23.875865 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:23.876131 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:23.877075 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:23.878036 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:23.879305 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:23.879783 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:23.879900 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:23.882880 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:23.882886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:23.882899 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:23.882927 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:23.885105 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:23.885662 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:23.885669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:23.885675 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:23.885831 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:23.886195 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:23.886199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:23.886200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:23.886720 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:23.886892 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:23.887426 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 12:29:23.887750 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":167,"threads_started":1}
I20260806 12:29:23.889573 14765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:23.895733 14765 catalog_manager.cc:1347] Generated new cluster ID: 5ac96c78fd7a49ea83d3c6aeec8b1b9d
I20260806 12:29:23.895803 14765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:23.913100 14765 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:23.914038 14765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:23.921800 14765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Generated new TSK 0
I20260806 12:29:23.922629 14765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab.
Entry for principal oryx/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab.
I20260806 12:29:23.961714 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45863
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:24.090845 14783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:24.091084 14783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:24.091194 14783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:24.095201 14783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:24.095252 14783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:24.095289 14783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:29:24.095321 14783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:24.100536 14783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:24.101470 14783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:24.102407 14783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:24.112573 14788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:24.112529 14791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:24.112582 14789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:24.112727 14783 server_base.cc:1034] running on GCE node
I20260806 12:29:24.113233 14783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:24.113912 14783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:24.115149 14783 hybrid_clock.cc:648] HybridClock initialized: now 1786019364115071 us; error 45 us; skew 500 ppm
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:24.117933 14783 init.cc:377] Logged in from keytab as oryx/127.6.152.129@KRBTEST.COM (short username oryx)
I20260806 12:29:24.118811 14783 webserver.cc:460] Webserver started at http://127.6.152.129:45991/ using document root <none> and password file <none>
I20260806 12:29:24.119443 14783 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:24.119511 14783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:24.119740 14783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:24.121742 14783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "1160f9bf01dc45d4bdfe9447b3519d70"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "98316c86e7d103a27f35fa1d9eca992a"
server_key_iv: "780400547682b576087b86dc5b4379b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.122329 14783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "1160f9bf01dc45d4bdfe9447b3519d70"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "98316c86e7d103a27f35fa1d9eca992a"
server_key_iv: "780400547682b576087b86dc5b4379b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.126125 14783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:29:24.128582 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:24.129791 14783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:24.129956 14783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "1160f9bf01dc45d4bdfe9447b3519d70"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "98316c86e7d103a27f35fa1d9eca992a"
server_key_iv: "780400547682b576087b86dc5b4379b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.130077 14783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:24.155612 14783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:24.158536 14783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:24.158748 14783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:24.160161 14783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:24.160238 14783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:24.160310 14783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:24.160354 14783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:24.171367 14783 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:39713
I20260806 12:29:24.171483 14920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:39713 every 8 connection(s)
I20260806 12:29:24.172246 14783 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:24.174767 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.161034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:44425 (local address 127.6.152.190:45863)
0806 12:29:24.161162 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.161166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.162158 (+ 992us) server_negotiation.cc:407] Connection header received
0806 12:29:24.163278 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.163282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.163349 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.163434 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.165223 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.166140 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.167747 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.168018 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.168137 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.170803 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.170809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.170812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.170849 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.172596 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.173193 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.173197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.173199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.173290 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.173637 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.173642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.173644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.173832 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.173959 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.174435 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.174578 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:29:24.175513 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.161560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.6.152.129:44425)
0806 12:29:24.162018 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.162059 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.163006 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.163923 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.163949 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.164422 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.165038 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.165056 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.166272 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.166276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.167619 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.167629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.168194 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.168199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.168412 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.168987 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.169003 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.170620 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.172731 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.172737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.172750 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.173068 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.173385 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.173389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.173391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.173536 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.173960 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.173968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.174193 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.174367 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.174700 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260806 12:29:24.176661 14921 heartbeater.cc:347] Connected to a master server at 127.6.152.190:45863
I20260806 12:29:24.176816 14921 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:24.177271 14921 heartbeater.cc:511] Master 127.6.152.190:45863 requested a full tablet report, sending...
I20260806 12:29:24.178611 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 14783
I20260806 12:29:24.178772 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:24.178803 14704 ts_manager.cc:194] Registered new tserver with Master: 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713)
I20260806 12:29:24.179201 6754 external_mini_cluster.cc:1428] Setting key b21b46accdfb2988551fd037b4e0b300
I20260806 12:29:24.180366 14704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:44425
I20260806 12:29:24.187201 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.178753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.6.152.129:33147)
0806 12:29:24.179083 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.179099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.179207 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.179514 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.179517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.179718 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.179847 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.179854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.180881 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.180887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.181720 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.181733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.182460 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.182463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.182615 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.183211 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.183226 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.183584 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.185512 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.185517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.185519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.185744 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.186039 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.186045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.186048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.186139 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.186686 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.186692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.186795 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.186897 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.186997 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":12,"thread_start_us":106,"threads_started":1}
I20260806 12:29:24.187372 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.179087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:33147 (local address 127.6.152.190:45863)
0806 12:29:24.179203 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.179207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.179220 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:24.179348 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.179351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.179392 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.179472 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.180026 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.180722 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.181873 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.182225 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.182304 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.183772 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.183777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.183781 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.183828 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.185403 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.185856 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.185859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.185861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.185945 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.186274 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.186281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.186285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.186552 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.186704 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.186998 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.187151 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for oryx/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab.
Entry for principal oryx/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab.
I20260806 12:29:24.209748 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45863
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:24.326117 14931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:24.326340 14931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:24.326416 14931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:24.330440 14931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:24.330494 14931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:24.330555 14931 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:29:24.330616 14931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:24.335974 14931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:24.336971 14931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:24.338059 14931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:24.347729 14939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:24.347738 14936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:24.348011 14937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:24.348189 14931 server_base.cc:1034] running on GCE node
I20260806 12:29:24.348572 14931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:24.349185 14931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:24.350373 14931 hybrid_clock.cc:648] HybridClock initialized: now 1786019364350352 us; error 33 us; skew 500 ppm
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:24.352979 14931 init.cc:377] Logged in from keytab as oryx/127.6.152.130@KRBTEST.COM (short username oryx)
I20260806 12:29:24.353785 14931 webserver.cc:460] Webserver started at http://127.6.152.130:46457/ using document root <none> and password file <none>
I20260806 12:29:24.354314 14931 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:24.354406 14931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:24.354650 14931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:24.356691 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "a52ab78a7f0d45dbbf9291061c0cba76"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "656f809cd3edabbb454149e6f63cd3c1"
server_key_iv: "0ca1604296af7eae3ed636a9fee1e233"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.357241 14931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "a52ab78a7f0d45dbbf9291061c0cba76"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "656f809cd3edabbb454149e6f63cd3c1"
server_key_iv: "0ca1604296af7eae3ed636a9fee1e233"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.360787 14931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:24.363206 14947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:24.364372 14931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:24.364508 14931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "a52ab78a7f0d45dbbf9291061c0cba76"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "656f809cd3edabbb454149e6f63cd3c1"
server_key_iv: "0ca1604296af7eae3ed636a9fee1e233"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.364627 14931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:24.386103 14931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:24.389021 14931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:24.389243 14931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:24.390779 14931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:24.390854 14931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:24.390931 14931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:24.390982 14931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:24.401826 14931 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:37111
I20260806 12:29:24.401867 15066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:37111 every 8 connection(s)
I20260806 12:29:24.402972 14931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:24.405275 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.391569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:57287 (local address 127.6.152.190:45863)
0806 12:29:24.391712 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.391717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.392642 (+ 925us) server_negotiation.cc:407] Connection header received
0806 12:29:24.393782 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.393786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.393863 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.393954 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.395842 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.396701 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.398676 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.398900 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.398981 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.401791 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.401795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.401797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.401825 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.403359 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.403959 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.403963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.403964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.404064 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.404390 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.404392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.404393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.404563 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.404645 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.405066 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.405157 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:29:24.405767 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 14931
I20260806 12:29:24.405907 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:24.406167 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.392020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.6.152.130:57287)
0806 12:29:24.392474 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.392524 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.393493 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.394441 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.394449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.394997 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.395679 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.395697 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.396872 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.396877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.398512 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.398527 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.399078 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.399083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.399304 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.400065 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.400084 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.401590 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.403481 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.403487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.403500 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.403835 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.404169 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.404172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.404174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.404308 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.404658 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.404663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.404861 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.405007 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.405228 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":130,"threads_started":1}
I20260806 12:29:24.406229 6754 external_mini_cluster.cc:1428] Setting key 4f45aab6f9c781916f6b63ccdc16f9eb
I20260806 12:29:24.407516 15067 heartbeater.cc:347] Connected to a master server at 127.6.152.190:45863
I20260806 12:29:24.407697 15067 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:24.408213 15067 heartbeater.cc:511] Master 127.6.152.190:45863 requested a full tablet report, sending...
I20260806 12:29:24.409299 14704 ts_manager.cc:194] Registered new tserver with Master: a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111)
I20260806 12:29:24.410072 14704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:57287
I20260806 12:29:24.417244 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.409759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:60625 (local address 127.6.152.190:45863)
0806 12:29:24.409854 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.409858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.410170 (+ 312us) server_negotiation.cc:407] Connection header received
0806 12:29:24.410283 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.410286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.410352 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.410424 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.411104 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.411813 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.412640 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.413051 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.413179 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.414127 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.414134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.414138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.414169 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.415637 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.416106 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.416108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.416109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.416179 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.416481 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.416483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.416484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.416667 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.416742 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.417055 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.417141 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 12:29:24.417356 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.409690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.6.152.130:60625)
0806 12:29:24.410052 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.410065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.410206 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.410533 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.410536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.410772 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.410883 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.410890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.411942 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.411945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.412523 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.412532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.413166 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.413168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.413235 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.413694 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.413706 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.413987 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.415748 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.415752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.415754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.416004 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.416273 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.416276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.416278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.416374 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.416771 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.416776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.416885 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.417004 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.417147 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for oryx/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab.
Entry for principal oryx/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab.
I20260806 12:29:24.437729 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45863
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:24.548267 15077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:24.548552 15077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:24.548642 15077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:24.552619 15077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:24.552693 15077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:24.552757 15077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 12:29:24.552830 15077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:24.557911 15077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.152.190:45863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:24.558915 15077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:24.559985 15077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:24.569264 15085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:24.569274 15082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:24.569272 15083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:24.569499 15077 server_base.cc:1034] running on GCE node
I20260806 12:29:24.569928 15077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:24.570549 15077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:24.571702 15077 hybrid_clock.cc:648] HybridClock initialized: now 1786019364571680 us; error 34 us; skew 500 ppm
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:24.574404 15077 init.cc:377] Logged in from keytab as oryx/127.6.152.131@KRBTEST.COM (short username oryx)
I20260806 12:29:24.575251 15077 webserver.cc:460] Webserver started at http://127.6.152.131:43465/ using document root <none> and password file <none>
I20260806 12:29:24.575729 15077 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:24.575784 15077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:24.575948 15077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:24.577955 15077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "d37f7c11d091484485d10c4221cd9a7a"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "7217e1c796bd733a7a00cfa21bba31c0"
server_key_iv: "589c84ac680fce32c6e2980ebc3027fa"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.578469 15077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "d37f7c11d091484485d10c4221cd9a7a"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "7217e1c796bd733a7a00cfa21bba31c0"
server_key_iv: "589c84ac680fce32c6e2980ebc3027fa"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.582053 15077 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 12:29:24.584507 15092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:24.585755 15077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:24.585850 15077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "d37f7c11d091484485d10c4221cd9a7a"
format_stamp: "Formatted at 2026-08-06 12:29:24 on dist-test-slave-1xrh"
server_key: "7217e1c796bd733a7a00cfa21bba31c0"
server_key_iv: "589c84ac680fce32c6e2980ebc3027fa"
server_key_version: "encryptionkey@0"
I20260806 12:29:24.585929 15077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:24.600330 15077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:24.603493 15077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:24.603644 15077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:24.604938 15077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:24.605026 15077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:24.605110 15077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:24.605154 15077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:24.616436 15077 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:45313
I20260806 12:29:24.616626 15212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:45313 every 8 connection(s)
I20260806 12:29:24.617305 15077 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:24.619709 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.605885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:60147 (local address 127.6.152.190:45863)
0806 12:29:24.606012 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.606016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.607011 (+ 995us) server_negotiation.cc:407] Connection header received
0806 12:29:24.608228 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.608232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.608296 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.608423 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.610229 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.611078 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.612691 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.613048 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.613167 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.615926 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.615929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.615932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.615959 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.617700 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.618314 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.618317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.618318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.618395 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.618691 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.618693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.618694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.618916 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.618995 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.619493 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.619588 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:29:24.620466 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.606375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.6.152.131:60147)
0806 12:29:24.606865 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.606905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.607973 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.608888 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.608898 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.609394 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.610030 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.610053 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.611254 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.611258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.612564 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.612574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.613210 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.613214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.613392 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.613970 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.613986 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.615761 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.617838 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.617843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.617856 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.618219 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.618459 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.618461 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.618462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.618594 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.619008 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.619013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.619286 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.619438 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.619666 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":139,"threads_started":1}
I20260806 12:29:24.621724 15213 heartbeater.cc:347] Connected to a master server at 127.6.152.190:45863
I20260806 12:29:24.621922 15213 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:24.622407 15213 heartbeater.cc:511] Master 127.6.152.190:45863 requested a full tablet report, sending...
I20260806 12:29:24.623545 14703 ts_manager.cc:194] Registered new tserver with Master: d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:24.623961 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15077
I20260806 12:29:24.624109 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:24.624257 14703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:60147
I20260806 12:29:24.624526 6754 external_mini_cluster.cc:1428] Setting key 583dcbedbc975910502ae58831901bea
I20260806 12:29:24.627174 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:29:24.632331 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.624772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:41737 (local address 127.6.152.190:45863)
0806 12:29:24.624884 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.624889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.625108 (+ 219us) server_negotiation.cc:407] Connection header received
0806 12:29:24.625297 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.625300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.625347 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.625434 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.626024 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.626755 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.627783 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.628010 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.628122 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.629203 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.629207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.629209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.629233 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.630661 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.631098 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.631102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.631103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.631232 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.631538 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.631541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.631543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.631711 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.631821 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.632078 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.632178 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:29:24.632361 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.624724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.6.152.131:41737)
0806 12:29:24.625032 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.625047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.625183 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.625535 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.625537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.625769 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.625900 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.625910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.626880 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.626905 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.627654 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.627664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.628128 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.628131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.628221 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.628726 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.628739 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.629058 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.630795 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.630798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.630800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.630993 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.631335 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.631338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.631340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.631429 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.631812 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.631815 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.631915 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.632026 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.632174 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":112,"threads_started":1}
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:29:24.652909 14759 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:24.646926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.6.152.190:45863)
0806 12:29:24.647053 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.647057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.647068 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:29:24.647100 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.647102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.647199 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.647286 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.648010 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.648512 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.649391 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.649607 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.649712 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.652774 (+ 3062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52704: BlockingRecv error: recv got EOF from 127.0.0.1:52704 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019364, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:24.662352 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.654331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.6.152.190:45863)
0806 12:29:24.654433 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.654437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.654636 (+ 199us) server_negotiation.cc:407] Connection header received
0806 12:29:24.654751 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.654753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.654787 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.654851 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.655495 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.655952 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.656778 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.656971 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.657047 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.659429 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.659432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.659434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.659455 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.660863 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.661260 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.661263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.661264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.661333 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.661657 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.661659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.661660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.661816 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.661919 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.662157 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.662243 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 12:29:24.665299 14703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:24.668488 14703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:24.683043 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.677097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.6.152.131:45313)
0806 12:29:24.677520 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.677534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.677951 (+ 417us) server_negotiation.cc:407] Connection header received
0806 12:29:24.678206 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.678210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.678416 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.678545 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.678943 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.679793 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.681228 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.682078 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.682159 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.682313 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.682760 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.682867 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":103,"threads_started":1}
I20260806 12:29:24.683097 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.677477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:37111 (local address 127.0.0.1:40422)
0806 12:29:24.677898 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.677919 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.677993 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.678996 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.678999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.679021 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.679103 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.679108 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.680511 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.680515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.681715 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.681725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.682592 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.682596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.682691 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.682715 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.682800 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.682938 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":25,"thread_start_us":129,"threads_started":1}
I20260806 12:29:24.683307 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.677886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.6.152.130:37111)
0806 12:29:24.678496 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.678508 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.678530 (+ 22us) server_negotiation.cc:407] Connection header received
0806 12:29:24.678610 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.678615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.678793 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.679008 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.679226 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.680396 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.681859 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.682444 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.682578 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.682745 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.682862 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.682960 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":112,"threads_started":1}
I20260806 12:29:24.683946 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.676876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.6.152.129:39713)
0806 12:29:24.677384 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.677394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.678030 (+ 636us) server_negotiation.cc:407] Connection header received
0806 12:29:24.678123 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.678129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.678366 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.678522 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.678992 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.680258 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.681901 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.682702 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.682797 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.682919 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.683700 (+ 781us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.683813 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":78,"threads_started":1}
I20260806 12:29:24.683434 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.676703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39713 (local address 127.0.0.1:38924)
0806 12:29:24.677625 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.677730 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.677872 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.678696 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.678702 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.678737 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.678849 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.678860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.680395 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.680398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.681757 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.681768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.682982 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.682986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.683085 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.683114 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.683221 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.683305 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":64,"spinlock_wait_cycles":2432,"thread_start_us":75,"threads_started":1}
I20260806 12:29:24.684379 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.676896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:45313 (local address 127.0.0.1:57648)
0806 12:29:24.677853 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.677866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.678045 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.678698 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.678705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.678734 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.678824 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.678828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.679908 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.679915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.681087 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.681098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.682411 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.682415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.682536 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.682577 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.683537 (+ 960us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.683612 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":6,"spinlock_wait_cycles":6144,"thread_start_us":97,"threads_started":1}
I20260806 12:29:24.686582 14999 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4ab1b525fb47ccab2dfa3849cdc303 (DEFAULT_TABLE table=test-table [id=4726e6a52d6749ce973d2a5d0b8001d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:24.686923 15145 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4ab1b525fb47ccab2dfa3849cdc303 (DEFAULT_TABLE table=test-table [id=4726e6a52d6749ce973d2a5d0b8001d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:24.687042 14852 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4ab1b525fb47ccab2dfa3849cdc303 (DEFAULT_TABLE table=test-table [id=4726e6a52d6749ce973d2a5d0b8001d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:24.687800 14999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4ab1b525fb47ccab2dfa3849cdc303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.687901 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4ab1b525fb47ccab2dfa3849cdc303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.687976 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4ab1b525fb47ccab2dfa3849cdc303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.694749 15242 tablet_bootstrap.cc:492] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Bootstrap starting.
I20260806 12:29:24.696074 15244 tablet_bootstrap.cc:492] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Bootstrap starting.
I20260806 12:29:24.696828 15242 tablet_bootstrap.cc:654] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.697535 15242 log.cc:824] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:24.697722 15244 tablet_bootstrap.cc:654] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.698364 15244 log.cc:824] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:24.698946 15243 tablet_bootstrap.cc:492] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Bootstrap starting.
I20260806 12:29:24.699534 15242 tablet_bootstrap.cc:492] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: No bootstrap required, opened a new log
I20260806 12:29:24.699723 15242 ts_tablet_manager.cc:1397] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:29:24.700347 15244 tablet_bootstrap.cc:492] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: No bootstrap required, opened a new log
I20260806 12:29:24.700528 15244 ts_tablet_manager.cc:1397] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 12:29:24.700812 15243 tablet_bootstrap.cc:654] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.701438 15243 log.cc:824] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:24.702925 15243 tablet_bootstrap.cc:492] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: No bootstrap required, opened a new log
I20260806 12:29:24.703157 15243 ts_tablet_manager.cc:1397] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 12:29:24.703636 15242 raft_consensus.cc:348] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.703917 15242 raft_consensus.cc:374] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.703984 15242 raft_consensus.cc:729] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37f7c11d091484485d10c4221cd9a7a, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.704222 15244 raft_consensus.cc:348] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.704453 15244 raft_consensus.cc:374] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.704414 15242 consensus_queue.cc:260] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.704504 15244 raft_consensus.cc:729] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.704900 15244 consensus_queue.cc:260] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.705395 15242 ts_tablet_manager.cc:1428] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:29:24.705559 15067 heartbeater.cc:503] Master 127.6.152.190:45863 was elected leader, sending a full tablet report...
I20260806 12:29:24.705696 15213 heartbeater.cc:503] Master 127.6.152.190:45863 was elected leader, sending a full tablet report...
I20260806 12:29:24.705783 15244 ts_tablet_manager.cc:1428] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:29:24.707088 15243 raft_consensus.cc:348] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.707347 15243 raft_consensus.cc:374] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.707422 15243 raft_consensus.cc:729] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.707865 15243 consensus_queue.cc:260] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.708498 14921 heartbeater.cc:503] Master 127.6.152.190:45863 was elected leader, sending a full tablet report...
I20260806 12:29:24.708755 15243 ts_tablet_manager.cc:1428] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260806 12:29:24.729027 15250 raft_consensus.cc:482] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:24.729207 15250 raft_consensus.cc:504] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.730315 15250 leader_election.cc:288] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313), a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111)
I20260806 12:29:24.735823 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.730602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:34665 (local address 127.6.152.131:45313)
0806 12:29:24.730751 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.730757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.731168 (+ 411us) server_negotiation.cc:407] Connection header received
0806 12:29:24.731304 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.731307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.731356 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.731444 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.731874 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.732664 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.734438 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.735098 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.735244 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.735297 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.735624 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.735696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:29:24.735823 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.730783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:33681 (local address 127.6.152.130:37111)
0806 12:29:24.730906 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.730909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.731187 (+ 278us) server_negotiation.cc:407] Connection header received
0806 12:29:24.731304 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.731307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.731369 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.731444 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.731874 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.732668 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.734438 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.735104 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.735258 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.735299 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.735624 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.735696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:29:24.735961 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.730502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:45313 (local address 127.6.152.129:34665)
0806 12:29:24.731030 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.731046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.731193 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.731470 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.731473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.731512 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.731752 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.731760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.732806 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.732812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.734294 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.734306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.735238 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.735242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.735451 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.735514 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.735590 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.735734 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":280,"spinlock_wait_cycles":27136,"thread_start_us":123,"threads_started":1}
I20260806 12:29:24.736279 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.730572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:37111 (local address 127.6.152.129:33681)
0806 12:29:24.731100 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.731113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.731203 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.731470 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.731473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.731513 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.731752 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.731760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.732789 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.732793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.734302 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.734307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.735239 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.735242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.735436 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.735507 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.735590 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.735735 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":257,"spinlock_wait_cycles":9472,"thread_start_us":125,"threads_started":1}
I20260806 12:29:24.736610 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4ab1b525fb47ccab2dfa3849cdc303" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a" is_pre_election: true
I20260806 12:29:24.736939 15165 raft_consensus.cc:2454] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 0.
I20260806 12:29:24.736919 15019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4ab1b525fb47ccab2dfa3849cdc303" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" is_pre_election: true
I20260806 12:29:24.737164 15019 raft_consensus.cc:2454] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 0.
I20260806 12:29:24.737481 14800 leader_election.cc:302] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1160f9bf01dc45d4bdfe9447b3519d70, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:24.737775 15250 raft_consensus.cc:2781] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:24.737888 15250 raft_consensus.cc:482] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:24.737942 15250 raft_consensus.cc:3037] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.739092 15250 raft_consensus.cc:504] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.739496 15250 leader_election.cc:288] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 election: Requested vote from peers d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313), a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111)
I20260806 12:29:24.739807 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4ab1b525fb47ccab2dfa3849cdc303" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a"
I20260806 12:29:24.739869 15019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4ab1b525fb47ccab2dfa3849cdc303" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a52ab78a7f0d45dbbf9291061c0cba76"
I20260806 12:29:24.739946 15165 raft_consensus.cc:3037] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.739997 15019 raft_consensus.cc:3037] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.740980 15165 raft_consensus.cc:2454] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 1.
I20260806 12:29:24.741262 14800 leader_election.cc:302] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1160f9bf01dc45d4bdfe9447b3519d70, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:24.741446 15250 raft_consensus.cc:2781] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:24.741609 15019 raft_consensus.cc:2454] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 1.
I20260806 12:29:24.741703 15250 raft_consensus.cc:686] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 LEADER]: Becoming Leader. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Running, Role: LEADER
I20260806 12:29:24.742041 15250 consensus_queue.cc:237] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } }
I20260806 12:29:24.745198 14703 catalog_manager.cc:5495] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:24.760993 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.757932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.6.152.129:39713)
0806 12:29:24.758050 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.758053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.758066 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:24.758141 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.758143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.758190 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.758267 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:24.758488 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.758967 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.759809 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.760004 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.760089 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.760312 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:24.760688 (+ 376us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:24.760771 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.760851 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.760895 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:29:24.766357 15019 raft_consensus.cc:1261] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Refusing update from remote peer 1160f9bf01dc45d4bdfe9447b3519d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:24.766857 15165 raft_consensus.cc:1261] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Refusing update from remote peer 1160f9bf01dc45d4bdfe9447b3519d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:24.767091 15254 consensus_queue.cc:1035] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:24.767480 15250 consensus_queue.cc:1035] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:24.774358 15259 mvcc.cc:205] Tried to move back new op lower bound from 7315535318076456960 to 7315535317985988608. Current Snapshot: MvccSnapshot[applied={T|T < 7315535318076456960}]
I20260806 12:29:24.782109 14704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:24.782248 14704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:24.784673 14704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Sending DeleteTablet for 3 replicas of tablet 4f4ab1b525fb47ccab2dfa3849cdc303
I20260806 12:29:24.785429 14852 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4ab1b525fb47ccab2dfa3849cdc303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:24 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38924
I20260806 12:29:24.785466 14999 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4ab1b525fb47ccab2dfa3849cdc303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:24 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:40422
I20260806 12:29:24.785597 15145 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4ab1b525fb47ccab2dfa3849cdc303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:24 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:57648
I20260806 12:29:24.786199 15266 tablet_replica.cc:330] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: stopping tablet replica
I20260806 12:29:24.786388 15267 tablet_replica.cc:330] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: stopping tablet replica
I20260806 12:29:24.786396 14704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 12:29:24.786509 15266 raft_consensus.cc:2229] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:24.786612 15267 raft_consensus.cc:2229] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 12:29:24.786799 14704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:24.786880 15267 raft_consensus.cc:2258] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:24.786909 15266 raft_consensus.cc:2258] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:24.787250 15268 tablet_replica.cc:330] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: stopping tablet replica
I20260806 12:29:24.787526 15268 raft_consensus.cc:2229] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:24.787788 15268 raft_consensus.cc:2258] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:24.788805 15266 ts_tablet_manager.cc:1905] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:24.789273 15267 ts_tablet_manager.cc:1905] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:24.791203 15268 ts_tablet_manager.cc:1905] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:24.792214 14852 tablet_service.cc:1467] Processing CreateTablet for tablet c4e7636dc72b47e3ae2c468c1ecae07f (DEFAULT_TABLE table=test-table [id=eb2f803c50224590b36947ae601e7b4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:24.792491 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e7636dc72b47e3ae2c468c1ecae07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.792596 14999 tablet_service.cc:1467] Processing CreateTablet for tablet c4e7636dc72b47e3ae2c468c1ecae07f (DEFAULT_TABLE table=test-table [id=eb2f803c50224590b36947ae601e7b4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:24.792649 15145 tablet_service.cc:1467] Processing CreateTablet for tablet c4e7636dc72b47e3ae2c468c1ecae07f (DEFAULT_TABLE table=test-table [id=eb2f803c50224590b36947ae601e7b4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:24.792845 14999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e7636dc72b47e3ae2c468c1ecae07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.792972 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4e7636dc72b47e3ae2c468c1ecae07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.793826 15268 ts_tablet_manager.cc:1918] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:24.793915 15268 log.cc:1196] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/4f4ab1b525fb47ccab2dfa3849cdc303
I20260806 12:29:24.794230 15268 ts_tablet_manager.cc:1939] T 4f4ab1b525fb47ccab2dfa3849cdc303 P d37f7c11d091484485d10c4221cd9a7a: Deleting consensus metadata
I20260806 12:29:24.794883 15243 tablet_bootstrap.cc:492] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Bootstrap starting.
I20260806 12:29:24.795241 15242 tablet_bootstrap.cc:492] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Bootstrap starting.
I20260806 12:29:24.795712 15244 tablet_bootstrap.cc:492] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Bootstrap starting.
I20260806 12:29:24.795912 15243 tablet_bootstrap.cc:654] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.796180 15242 tablet_bootstrap.cc:654] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.796663 15244 tablet_bootstrap.cc:654] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.796964 14689 catalog_manager.cc:4841] TS d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313): tablet 4f4ab1b525fb47ccab2dfa3849cdc303 (table test-table [id=4726e6a52d6749ce973d2a5d0b8001d1]) successfully deleted
I20260806 12:29:24.797209 15243 tablet_bootstrap.cc:492] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: No bootstrap required, opened a new log
I20260806 12:29:24.797278 15243 ts_tablet_manager.cc:1397] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:24.797716 15243 raft_consensus.cc:348] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.797819 15243 raft_consensus.cc:374] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.797850 15243 raft_consensus.cc:729] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.797964 15243 consensus_queue.cc:260] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.798213 15243 ts_tablet_manager.cc:1428] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:24.798422 15266 ts_tablet_manager.cc:1918] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:24.798520 15266 log.cc:1196] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/4f4ab1b525fb47ccab2dfa3849cdc303
I20260806 12:29:24.798861 15266 ts_tablet_manager.cc:1939] T 4f4ab1b525fb47ccab2dfa3849cdc303 P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting consensus metadata
I20260806 12:29:24.799908 14691 catalog_manager.cc:4841] TS 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713): tablet 4f4ab1b525fb47ccab2dfa3849cdc303 (table test-table [id=4726e6a52d6749ce973d2a5d0b8001d1]) successfully deleted
I20260806 12:29:24.800614 15267 ts_tablet_manager.cc:1918] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:24.800711 15267 log.cc:1196] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/4f4ab1b525fb47ccab2dfa3849cdc303
I20260806 12:29:24.800829 15242 tablet_bootstrap.cc:492] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: No bootstrap required, opened a new log
I20260806 12:29:24.800916 15242 ts_tablet_manager.cc:1397] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 12:29:24.801021 15267 ts_tablet_manager.cc:1939] T 4f4ab1b525fb47ccab2dfa3849cdc303 P a52ab78a7f0d45dbbf9291061c0cba76: Deleting consensus metadata
I20260806 12:29:24.801403 15242 raft_consensus.cc:348] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.801518 15242 raft_consensus.cc:374] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.801551 15242 raft_consensus.cc:729] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37f7c11d091484485d10c4221cd9a7a, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.801685 15242 consensus_queue.cc:260] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.801962 15242 ts_tablet_manager.cc:1428] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:24.802255 14688 catalog_manager.cc:4841] TS a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111): tablet 4f4ab1b525fb47ccab2dfa3849cdc303 (table test-table [id=4726e6a52d6749ce973d2a5d0b8001d1]) successfully deleted
I20260806 12:29:24.803279 15244 tablet_bootstrap.cc:492] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: No bootstrap required, opened a new log
I20260806 12:29:24.803366 15244 ts_tablet_manager.cc:1397] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 12:29:24.803742 15244 raft_consensus.cc:348] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.803846 15244 raft_consensus.cc:374] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.803876 15244 raft_consensus.cc:729] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.804018 15244 consensus_queue.cc:260] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.804266 15244 ts_tablet_manager.cc:1428] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:29:24.848978 15249 raft_consensus.cc:482] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:24.849130 15249 raft_consensus.cc:504] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.850322 15249 leader_election.cc:288] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:24.854992 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.850547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39713 (local address 127.6.152.130:48121)
0806 12:29:24.850726 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.850742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.850824 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.851248 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.851252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.851270 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.851397 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.851403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.852438 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.852442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.853706 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.853718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.854559 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.854563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.854677 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.854709 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.854792 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.854861 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 12:29:24.855062 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.850635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:48121 (local address 127.6.152.129:39713)
0806 12:29:24.850772 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.850777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.850794 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:29:24.850964 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.850967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.851029 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.851095 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.851525 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.852315 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.853833 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.854432 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.854560 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.854608 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.854831 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.854923 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:24.855187 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.850954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:36397 (local address 127.6.152.131:45313)
0806 12:29:24.851086 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.851092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.851216 (+ 124us) server_negotiation.cc:407] Connection header received
0806 12:29:24.851371 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.851374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.851424 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.851507 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:24.851778 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.852473 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.853778 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.854432 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.854555 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.854608 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.854965 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.855035 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 12:29:24.855556 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.850569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:45313 (local address 127.6.152.130:36397)
0806 12:29:24.851078 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.851093 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.851260 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.851543 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.851546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.851562 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:24.851666 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.851672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.852607 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.852611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.853567 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.853577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.854709 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.854712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.854814 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.854841 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.855317 (+ 476us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.855392 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260806 12:29:24.855607 14873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e7636dc72b47e3ae2c468c1ecae07f" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" is_pre_election: true
I20260806 12:29:24.855877 14873 raft_consensus.cc:2454] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 0.
I20260806 12:29:24.856070 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e7636dc72b47e3ae2c468c1ecae07f" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a" is_pre_election: true
I20260806 12:29:24.856212 15165 raft_consensus.cc:2454] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 0.
I20260806 12:29:24.856353 14951 leader_election.cc:302] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1160f9bf01dc45d4bdfe9447b3519d70, a52ab78a7f0d45dbbf9291061c0cba76; no voters:
I20260806 12:29:24.856652 15249 raft_consensus.cc:2781] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:24.856717 15249 raft_consensus.cc:482] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:24.856755 15249 raft_consensus.cc:3037] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.858048 15249 raft_consensus.cc:504] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.858388 15249 leader_election.cc:288] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 election: Requested vote from peers 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:24.858700 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e7636dc72b47e3ae2c468c1ecae07f" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a"
I20260806 12:29:24.858763 14873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4e7636dc72b47e3ae2c468c1ecae07f" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160f9bf01dc45d4bdfe9447b3519d70"
I20260806 12:29:24.858840 15165 raft_consensus.cc:3037] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.858881 14873 raft_consensus.cc:3037] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.859850 15165 raft_consensus.cc:2454] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 1.
I20260806 12:29:24.860054 14873 raft_consensus.cc:2454] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 1.
I20260806 12:29:24.860146 14949 leader_election.cc:302] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a52ab78a7f0d45dbbf9291061c0cba76, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:24.860324 15249 raft_consensus.cc:2781] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:24.860590 15249 raft_consensus.cc:686] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 LEADER]: Becoming Leader. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Running, Role: LEADER
I20260806 12:29:24.860836 15249 consensus_queue.cc:237] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.863168 14703 catalog_manager.cc:5495] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 reported cstate change: term changed from 0 to 1, leader changed from <none> to a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } health_report { overall_health: UNKNOWN } } }
W20260806 12:29:24.868273 15214 tablet.cc:2369] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:29:24 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019363, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.190@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
W20260806 12:29:24.904587 15068 tablet.cc:2369] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:29:24.906766 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.897282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45863 (local address 127.0.0.1:52714)
0806 12:29:24.897564 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:24.897582 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:24.897704 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:24.897985 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:24.897987 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:24.898347 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:24.898438 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.898443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.899590 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.899593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.900125 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:24.900132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.900807 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.900809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:24.900868 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.901503 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:24.901514 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:24.903262 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:24.905230 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.905238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.905243 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.905452 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.905770 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:24.905772 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:24.905773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:24.905846 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:24.906293 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:24.906298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:24.906415 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 12:29:24.906515 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 12:29:24.906622 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":121,"threads_started":1}
I20260806 12:29:24.906922 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:24.897323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.6.152.190:45863)
0806 12:29:24.897471 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:24.897477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:24.897688 (+ 211us) server_negotiation.cc:407] Connection header received
0806 12:29:24.897799 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:24.897802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:24.897857 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:24.897968 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:24.898584 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.899482 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.900247 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:24.900691 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:24.900812 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:24.903409 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:24.903413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:24.903416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:24.903445 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:24.905117 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.905563 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.905567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.905569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.905667 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:24.905950 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:24.905953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:24.905955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:24.906187 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:24.906313 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:24.906618 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 12:29:24.906766 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:29:24.911257 14703 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:52714:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 12:29:24.911960 14703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:24.916889 14852 tablet_service.cc:1467] Processing CreateTablet for tablet 28a4373aa63a40739723355efc7e1953 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2f229cc5edae4db79e73072eeb2594f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:29:24.916889 14999 tablet_service.cc:1467] Processing CreateTablet for tablet 28a4373aa63a40739723355efc7e1953 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2f229cc5edae4db79e73072eeb2594f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:29:24.917290 14999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a4373aa63a40739723355efc7e1953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.917290 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a4373aa63a40739723355efc7e1953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.920054 15145 tablet_service.cc:1467] Processing CreateTablet for tablet 28a4373aa63a40739723355efc7e1953 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2f229cc5edae4db79e73072eeb2594f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 12:29:24.920437 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28a4373aa63a40739723355efc7e1953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:24.920600 15243 tablet_bootstrap.cc:492] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: Bootstrap starting.
I20260806 12:29:24.921581 15243 tablet_bootstrap.cc:654] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.923033 15243 tablet_bootstrap.cc:492] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: No bootstrap required, opened a new log
I20260806 12:29:24.923149 15243 ts_tablet_manager.cc:1397] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:29:24.923623 15243 raft_consensus.cc:348] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.923743 15243 raft_consensus.cc:374] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.923780 15243 raft_consensus.cc:729] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.923918 15243 consensus_queue.cc:260] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
W20260806 12:29:24.924151 14923 tablet.cc:2369] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:29:24.924280 15242 tablet_bootstrap.cc:492] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: Bootstrap starting.
I20260806 12:29:24.924547 15244 tablet_bootstrap.cc:492] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: Bootstrap starting.
I20260806 12:29:24.924701 15250 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:24.924817 15250 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:24.925331 15242 tablet_bootstrap.cc:654] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.925489 15244 tablet_bootstrap.cc:654] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:24.926434 15243 ts_tablet_manager.cc:1428] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 12:29:24.927472 15244 tablet_bootstrap.cc:492] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: No bootstrap required, opened a new log
I20260806 12:29:24.927582 15244 ts_tablet_manager.cc:1397] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:29:24.928072 15244 raft_consensus.cc:348] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.928190 15244 raft_consensus.cc:374] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.928236 15244 raft_consensus.cc:729] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.928344 15242 tablet_bootstrap.cc:492] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: No bootstrap required, opened a new log
I20260806 12:29:24.928428 15242 ts_tablet_manager.cc:1397] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 12:29:24.928373 15244 consensus_queue.cc:260] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.928768 15249 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:24.928915 15249 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:24.928884 15242 raft_consensus.cc:348] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.929021 15242 raft_consensus.cc:374] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:24.929065 15242 raft_consensus.cc:729] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37f7c11d091484485d10c4221cd9a7a, State: Initialized, Role: FOLLOWER
I20260806 12:29:24.929215 15242 consensus_queue.cc:260] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.929522 15242 ts_tablet_manager.cc:1428] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:29:24.929489 15248 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:24.929623 15248 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:24.928781 15244 ts_tablet_manager.cc:1428] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 12:29:24.978885 15249 raft_consensus.cc:482] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:24.979053 15249 raft_consensus.cc:504] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.979568 15249 leader_election.cc:288] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:24.980024 14873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a4373aa63a40739723355efc7e1953" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" is_pre_election: true
I20260806 12:29:24.980038 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a4373aa63a40739723355efc7e1953" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a" is_pre_election: true
I20260806 12:29:24.980199 14873 raft_consensus.cc:2454] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 0.
I20260806 12:29:24.980224 15165 raft_consensus.cc:2454] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 0.
I20260806 12:29:24.980520 14949 leader_election.cc:302] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a52ab78a7f0d45dbbf9291061c0cba76, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:24.980733 15249 raft_consensus.cc:2781] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:24.980809 15249 raft_consensus.cc:482] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:24.980851 15249 raft_consensus.cc:3037] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.981842 15249 raft_consensus.cc:504] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.982178 15249 leader_election.cc:288] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 election: Requested vote from peers 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:24.982522 14873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a4373aa63a40739723355efc7e1953" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160f9bf01dc45d4bdfe9447b3519d70"
I20260806 12:29:24.982575 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28a4373aa63a40739723355efc7e1953" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a"
I20260806 12:29:24.982687 14873 raft_consensus.cc:3037] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.982712 15165 raft_consensus.cc:3037] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:24.983736 15165 raft_consensus.cc:2454] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 1.
I20260806 12:29:24.983822 14873 raft_consensus.cc:2454] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 1.
I20260806 12:29:24.984058 14949 leader_election.cc:302] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a52ab78a7f0d45dbbf9291061c0cba76, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:24.984272 15249 raft_consensus.cc:2781] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:24.984392 15249 raft_consensus.cc:686] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 LEADER]: Becoming Leader. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Running, Role: LEADER
I20260806 12:29:24.984536 15249 consensus_queue.cc:237] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:24.985303 15273 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: TxnStatusTablet state changed. Reason: New leader a52ab78a7f0d45dbbf9291061c0cba76. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:24.985419 15273 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:29:24.986200 15285 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 12:29:24.986373 15285 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 12:29:24.986578 14704 catalog_manager.cc:5495] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 reported cstate change: term changed from 0 to 1, leader changed from <none> to a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:25.004840 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.000858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:37111 (local address 127.0.0.1:40424)
0806 12:29:25.001019 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:25.001035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:25.001137 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:25.001462 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:25.001464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:25.001470 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:29:25.001568 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:25.001573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.002562 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.002564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:25.003218 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:25.003228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.003783 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.003786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:25.003863 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.003906 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.004602 (+ 696us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.004614 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 12:29:25.004663 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 12:29:25.004730 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 12:29:25.004858 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.000942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.6.152.130:37111)
0806 12:29:25.001063 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.001067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.001084 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:29:25.001273 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.001277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.001341 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.001447 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:25.001669 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.002454 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.003328 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.003671 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.003768 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.004042 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.004511 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.004620 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:25.004678 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 12:29:25.004734 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:29:25.008260 15165 raft_consensus.cc:1261] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Refusing update from remote peer a52ab78a7f0d45dbbf9291061c0cba76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.008460 14873 raft_consensus.cc:1261] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Refusing update from remote peer a52ab78a7f0d45dbbf9291061c0cba76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.008769 15249 consensus_queue.cc:1035] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.009016 15273 consensus_queue.cc:1035] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.011071 15248 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: TxnStatusTablet state changed. Reason: New leader a52ab78a7f0d45dbbf9291061c0cba76. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:25.011062 15250 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: TxnStatusTablet state changed. Reason: New leader a52ab78a7f0d45dbbf9291061c0cba76. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:25.011217 15250 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:25.011219 15248 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:25.013782 15290 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:25.013957 15290 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:29:25.014195 15290 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:25.014346 15290 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P a52ab78a7f0d45dbbf9291061c0cba76: This TxnStatusTablet replica's current role is: LEADER
I20260806 12:29:25.014839 15257 mvcc.cc:205] Tried to move back new op lower bound from 7315535319068213248 to 7315535318978396160. Current Snapshot: MvccSnapshot[applied={T|T < 7315535319068213248}]
I20260806 12:29:25.015136 15250 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:25.015239 15250 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P 1160f9bf01dc45d4bdfe9447b3519d70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:25.015313 15258 mvcc.cc:205] Tried to move back new op lower bound from 7315535319068213248 to 7315535318978396160. Current Snapshot: MvccSnapshot[applied={T|T < 7315535319068213248 or (T in {7315535319068213248})}]
I20260806 12:29:25.017966 15248 tablet_replica.cc:439] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } } }
I20260806 12:29:25.018080 15248 tablet_replica.cc:442] T 28a4373aa63a40739723355efc7e1953 P d37f7c11d091484485d10c4221cd9a7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 12:29:25.022737 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.019442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.6.152.130:37111)
0806 12:29:25.019557 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.019561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.019765 (+ 204us) server_negotiation.cc:407] Connection header received
0806 12:29:25.019920 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.019923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.019966 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.020053 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:25.020308 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.020803 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.021633 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.021949 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.022056 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.022322 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.022397 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.022474 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:25.022566 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 12:29:25.022613 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:29:25.032729 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.029416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:39091 (local address 127.6.152.130:37111)
0806 12:29:25.029520 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.029524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.029699 (+ 175us) server_negotiation.cc:407] Connection header received
0806 12:29:25.029847 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.029850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.029891 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.029979 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:25.030229 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.030808 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.031726 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.031948 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.032050 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.032329 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.032388 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.032464 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:25.032550 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 12:29:25.032589 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 12:29:25.032737 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.029320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:37111 (local address 127.6.152.130:39091)
0806 12:29:25.029614 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:25.029626 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:25.029746 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:25.029986 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:25.029988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:25.029994 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 12:29:25.030127 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:25.030136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.030908 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.030914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:25.031611 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:25.031620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.032057 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.032060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:25.032151 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.032189 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.032465 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.032472 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 12:29:25.032537 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 12:29:25.032605 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":123,"threads_started":1}
I20260806 12:29:25.039414 15165 raft_consensus.cc:1261] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Refusing update from remote peer a52ab78a7f0d45dbbf9291061c0cba76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.039404 14873 raft_consensus.cc:1261] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Refusing update from remote peer a52ab78a7f0d45dbbf9291061c0cba76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.039785 15290 consensus_queue.cc:1035] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.040089 15290 consensus_queue.cc:1035] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.082176 14704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:25.082348 14704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:25.084362 14704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Sending DeleteTablet for 3 replicas of tablet c4e7636dc72b47e3ae2c468c1ecae07f
I20260806 12:29:25.084882 14852 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e7636dc72b47e3ae2c468c1ecae07f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38924
I20260806 12:29:25.085047 14998 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e7636dc72b47e3ae2c468c1ecae07f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:40422
I20260806 12:29:25.085259 15266 tablet_replica.cc:330] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: stopping tablet replica
I20260806 12:29:25.085394 15266 raft_consensus.cc:2229] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:25.085544 15266 raft_consensus.cc:2258] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.085731 15267 tablet_replica.cc:330] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: stopping tablet replica
I20260806 12:29:25.085879 15267 raft_consensus.cc:2229] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:25.086196 15267 raft_consensus.cc:2258] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.086381 15266 ts_tablet_manager.cc:1905] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.086710 15145 tablet_service.cc:1514] Processing DeleteTablet for tablet c4e7636dc72b47e3ae2c468c1ecae07f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:57648
I20260806 12:29:25.089447 15266 ts_tablet_manager.cc:1918] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:29:25.089520 15266 log.cc:1196] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/c4e7636dc72b47e3ae2c468c1ecae07f
I20260806 12:29:25.089835 15266 ts_tablet_manager.cc:1939] T c4e7636dc72b47e3ae2c468c1ecae07f P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting consensus metadata
I20260806 12:29:25.090531 14691 catalog_manager.cc:4841] TS 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713): tablet c4e7636dc72b47e3ae2c468c1ecae07f (table test-table [id=eb2f803c50224590b36947ae601e7b4a]) successfully deleted
I20260806 12:29:25.091041 15268 tablet_replica.cc:330] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: stopping tablet replica
I20260806 12:29:25.091180 15268 raft_consensus.cc:2229] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:25.091318 15268 raft_consensus.cc:2258] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.092439 15267 ts_tablet_manager.cc:1905] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.092459 15268 ts_tablet_manager.cc:1905] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.095099 15267 ts_tablet_manager.cc:1918] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:29:25.095099 15268 ts_tablet_manager.cc:1918] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:29:25.095222 15268 log.cc:1196] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/c4e7636dc72b47e3ae2c468c1ecae07f
I20260806 12:29:25.095222 15267 log.cc:1196] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/c4e7636dc72b47e3ae2c468c1ecae07f
I20260806 12:29:25.095476 15267 ts_tablet_manager.cc:1939] T c4e7636dc72b47e3ae2c468c1ecae07f P a52ab78a7f0d45dbbf9291061c0cba76: Deleting consensus metadata
I20260806 12:29:25.095476 15268 ts_tablet_manager.cc:1939] T c4e7636dc72b47e3ae2c468c1ecae07f P d37f7c11d091484485d10c4221cd9a7a: Deleting consensus metadata
I20260806 12:29:25.096125 14688 catalog_manager.cc:4841] TS a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111): tablet c4e7636dc72b47e3ae2c468c1ecae07f (table test-table [id=eb2f803c50224590b36947ae601e7b4a]) successfully deleted
I20260806 12:29:25.096153 14689 catalog_manager.cc:4841] TS d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313): tablet c4e7636dc72b47e3ae2c468c1ecae07f (table test-table [id=eb2f803c50224590b36947ae601e7b4a]) successfully deleted
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[14659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:29:25.115372 14759 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:25.108276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.6.152.190:45863)
0806 12:29:25.108426 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.108431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.108710 (+ 279us) server_negotiation.cc:407] Connection header received
0806 12:29:25.108834 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.108837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.108880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.108948 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:25.109797 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.110404 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.111702 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.111934 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.112016 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.115254 (+ 3238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52716: BlockingRecv error: recv got EOF from 127.0.0.1:52716 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[14659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM
I20260806 12:29:25.126106 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.116956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.6.152.190:45863)
0806 12:29:25.117054 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.117057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.117419 (+ 362us) server_negotiation.cc:407] Connection header received
0806 12:29:25.117482 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.117485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.117529 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.117616 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:25.118203 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.119002 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.120001 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.120307 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.120424 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.122606 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:25.122611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:25.122614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:25.122642 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:25.124486 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:25.124886 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:25.124889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:25.124893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:25.124989 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:25.125247 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:25.125250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:25.125252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:25.125475 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:25.125618 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:25.125816 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 12:29:25.125949 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 12:29:25.128876 14704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:25.129395 14704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:25.134714 14852 tablet_service.cc:1467] Processing CreateTablet for tablet 9a69442abbed436c8e0b71b34012bbbd (DEFAULT_TABLE table=test-table [id=9482e927e4914db295ff0e1d4c831836]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:25.134881 14998 tablet_service.cc:1467] Processing CreateTablet for tablet 9a69442abbed436c8e0b71b34012bbbd (DEFAULT_TABLE table=test-table [id=9482e927e4914db295ff0e1d4c831836]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:25.135073 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a69442abbed436c8e0b71b34012bbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:25.135268 14998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a69442abbed436c8e0b71b34012bbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:25.137722 15243 tablet_bootstrap.cc:492] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Bootstrap starting.
I20260806 12:29:25.137784 15244 tablet_bootstrap.cc:492] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Bootstrap starting.
I20260806 12:29:25.138432 15243 tablet_bootstrap.cc:654] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:25.138701 15244 tablet_bootstrap.cc:654] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:25.139742 15243 tablet_bootstrap.cc:492] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: No bootstrap required, opened a new log
I20260806 12:29:25.139832 15243 ts_tablet_manager.cc:1397] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:25.139781 15145 tablet_service.cc:1467] Processing CreateTablet for tablet 9a69442abbed436c8e0b71b34012bbbd (DEFAULT_TABLE table=test-table [id=9482e927e4914db295ff0e1d4c831836]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:25.140054 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a69442abbed436c8e0b71b34012bbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:25.140230 15244 tablet_bootstrap.cc:492] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: No bootstrap required, opened a new log
I20260806 12:29:25.140311 15244 ts_tablet_manager.cc:1397] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 12:29:25.140251 15243 raft_consensus.cc:348] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.140372 15243 raft_consensus.cc:374] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:25.140412 15243 raft_consensus.cc:729] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Initialized, Role: FOLLOWER
I20260806 12:29:25.140543 15243 consensus_queue.cc:260] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.140785 15243 ts_tablet_manager.cc:1428] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:25.140745 15244 raft_consensus.cc:348] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.140853 15244 raft_consensus.cc:374] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:25.140890 15244 raft_consensus.cc:729] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Initialized, Role: FOLLOWER
I20260806 12:29:25.141028 15244 consensus_queue.cc:260] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.141286 15244 ts_tablet_manager.cc:1428] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:25.142593 15242 tablet_bootstrap.cc:492] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Bootstrap starting.
I20260806 12:29:25.143501 15242 tablet_bootstrap.cc:654] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:25.144676 15242 tablet_bootstrap.cc:492] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: No bootstrap required, opened a new log
I20260806 12:29:25.144788 15242 ts_tablet_manager.cc:1397] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:25.145254 15242 raft_consensus.cc:348] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.145465 15242 raft_consensus.cc:374] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:25.145520 15242 raft_consensus.cc:729] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37f7c11d091484485d10c4221cd9a7a, State: Initialized, Role: FOLLOWER
I20260806 12:29:25.145642 15242 consensus_queue.cc:260] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.145902 15242 ts_tablet_manager.cc:1428] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:25.395061 15250 raft_consensus.cc:482] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:25.395213 15250 raft_consensus.cc:504] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.395612 15250 leader_election.cc:288] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:25.396071 15019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a69442abbed436c8e0b71b34012bbbd" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" is_pre_election: true
I20260806 12:29:25.396085 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a69442abbed436c8e0b71b34012bbbd" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a" is_pre_election: true
I20260806 12:29:25.396216 15019 raft_consensus.cc:2454] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 0.
I20260806 12:29:25.396268 15165 raft_consensus.cc:2454] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 0.
I20260806 12:29:25.396577 14799 leader_election.cc:302] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1160f9bf01dc45d4bdfe9447b3519d70, a52ab78a7f0d45dbbf9291061c0cba76; no voters:
I20260806 12:29:25.396791 15250 raft_consensus.cc:2781] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:25.396860 15250 raft_consensus.cc:482] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:25.396900 15250 raft_consensus.cc:3037] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:25.397884 15250 raft_consensus.cc:504] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.398247 15250 leader_election.cc:288] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 election: Requested vote from peers a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:25.398577 15019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a69442abbed436c8e0b71b34012bbbd" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a52ab78a7f0d45dbbf9291061c0cba76"
I20260806 12:29:25.398700 15019 raft_consensus.cc:3037] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:25.398689 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a69442abbed436c8e0b71b34012bbbd" candidate_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a"
I20260806 12:29:25.398890 15165 raft_consensus.cc:3037] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:25.399789 15019 raft_consensus.cc:2454] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 1.
I20260806 12:29:25.399933 15165 raft_consensus.cc:2454] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160f9bf01dc45d4bdfe9447b3519d70 in term 1.
I20260806 12:29:25.400092 14799 leader_election.cc:302] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1160f9bf01dc45d4bdfe9447b3519d70, a52ab78a7f0d45dbbf9291061c0cba76; no voters:
I20260806 12:29:25.400254 15250 raft_consensus.cc:2781] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:25.400425 15250 raft_consensus.cc:686] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 LEADER]: Becoming Leader. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Running, Role: LEADER
I20260806 12:29:25.400594 15250 consensus_queue.cc:237] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.402438 14703 catalog_manager.cc:5495] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:25.442796 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.438124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.6.152.129:39713)
0806 12:29:25.438767 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.438774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.438794 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:29:25.438868 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.438874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.438962 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.439055 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:25.439366 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.440336 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.441307 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.441690 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.441809 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.442210 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.442311 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.442403 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:25.442549 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 12:29:25.442607 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"spinlock_wait_cycles":591616,"thread_start_us":84,"threads_started":1}
I20260806 12:29:25.445271 15019 raft_consensus.cc:1261] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Refusing update from remote peer 1160f9bf01dc45d4bdfe9447b3519d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.445331 15165 raft_consensus.cc:1261] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Refusing update from remote peer 1160f9bf01dc45d4bdfe9447b3519d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.445864 15314 consensus_queue.cc:1035] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.446048 15250 consensus_queue.cc:1035] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.454537 14704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:25.454654 14704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:25.456601 14704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Sending DeleteTablet for 3 replicas of tablet 9a69442abbed436c8e0b71b34012bbbd
I20260806 12:29:25.457118 14998 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a69442abbed436c8e0b71b34012bbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:40422
I20260806 12:29:25.457180 15145 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a69442abbed436c8e0b71b34012bbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:57648
I20260806 12:29:25.457202 14852 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a69442abbed436c8e0b71b34012bbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38924
I20260806 12:29:25.457293 15267 tablet_replica.cc:330] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: stopping tablet replica
I20260806 12:29:25.457324 15268 tablet_replica.cc:330] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: stopping tablet replica
I20260806 12:29:25.457398 15267 raft_consensus.cc:2229] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:25.457407 15268 raft_consensus.cc:2229] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:25.457508 15268 raft_consensus.cc:2258] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.457516 15267 raft_consensus.cc:2258] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.457690 15266 tablet_replica.cc:330] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: stopping tablet replica
I20260806 12:29:25.457808 15266 raft_consensus.cc:2229] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:25.457826 14704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 12:29:25.458017 15266 raft_consensus.cc:2258] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 12:29:25.458226 14704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:25.458287 15268 ts_tablet_manager.cc:1905] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.458643 15267 ts_tablet_manager.cc:1905] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.460666 15268 ts_tablet_manager.cc:1918] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:25.460752 15268 log.cc:1196] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/9a69442abbed436c8e0b71b34012bbbd
I20260806 12:29:25.461051 15268 ts_tablet_manager.cc:1939] T 9a69442abbed436c8e0b71b34012bbbd P d37f7c11d091484485d10c4221cd9a7a: Deleting consensus metadata
I20260806 12:29:25.461061 15267 ts_tablet_manager.cc:1918] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:25.461130 15267 log.cc:1196] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/9a69442abbed436c8e0b71b34012bbbd
I20260806 12:29:25.461390 15267 ts_tablet_manager.cc:1939] T 9a69442abbed436c8e0b71b34012bbbd P a52ab78a7f0d45dbbf9291061c0cba76: Deleting consensus metadata
I20260806 12:29:25.461930 14689 catalog_manager.cc:4841] TS d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313): tablet 9a69442abbed436c8e0b71b34012bbbd (table test-table [id=9482e927e4914db295ff0e1d4c831836]) successfully deleted
I20260806 12:29:25.462047 14688 catalog_manager.cc:4841] TS a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111): tablet 9a69442abbed436c8e0b71b34012bbbd (table test-table [id=9482e927e4914db295ff0e1d4c831836]) successfully deleted
I20260806 12:29:25.462378 15266 ts_tablet_manager.cc:1905] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.462869 14852 tablet_service.cc:1467] Processing CreateTablet for tablet 37b47dfa331f4a4b9fb105aca3d06c9c (DEFAULT_TABLE table=test-table [id=e8209641252d43f98832027fb7e65460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:25.463156 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b47dfa331f4a4b9fb105aca3d06c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:25.463145 15145 tablet_service.cc:1467] Processing CreateTablet for tablet 37b47dfa331f4a4b9fb105aca3d06c9c (DEFAULT_TABLE table=test-table [id=e8209641252d43f98832027fb7e65460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:25.463392 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b47dfa331f4a4b9fb105aca3d06c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:25.464998 15243 tablet_bootstrap.cc:492] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: Bootstrap starting.
I20260806 12:29:25.465700 15242 tablet_bootstrap.cc:492] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Bootstrap starting.
I20260806 12:29:25.465874 15243 tablet_bootstrap.cc:654] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:25.466440 15242 tablet_bootstrap.cc:654] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:25.466526 14998 tablet_service.cc:1467] Processing CreateTablet for tablet 37b47dfa331f4a4b9fb105aca3d06c9c (DEFAULT_TABLE table=test-table [id=e8209641252d43f98832027fb7e65460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:25.466760 14998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b47dfa331f4a4b9fb105aca3d06c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:25.467640 15243 tablet_bootstrap.cc:492] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: No bootstrap required, opened a new log
I20260806 12:29:25.467720 15242 tablet_bootstrap.cc:492] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: No bootstrap required, opened a new log
I20260806 12:29:25.467730 15243 ts_tablet_manager.cc:1397] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 12:29:25.467798 15242 ts_tablet_manager.cc:1397] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:25.468132 15243 raft_consensus.cc:348] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.468199 15242 raft_consensus.cc:348] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.468274 15243 raft_consensus.cc:374] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:25.468289 15242 raft_consensus.cc:374] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:25.468324 15242 raft_consensus.cc:729] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37f7c11d091484485d10c4221cd9a7a, State: Initialized, Role: FOLLOWER
I20260806 12:29:25.468324 15243 raft_consensus.cc:729] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160f9bf01dc45d4bdfe9447b3519d70, State: Initialized, Role: FOLLOWER
I20260806 12:29:25.468461 15243 consensus_queue.cc:260] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.468461 15242 consensus_queue.cc:260] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.468724 15243 ts_tablet_manager.cc:1428] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:25.468724 15242 ts_tablet_manager.cc:1428] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 12:29:25.468915 15266 ts_tablet_manager.cc:1918] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:25.468943 15244 tablet_bootstrap.cc:492] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Bootstrap starting.
I20260806 12:29:25.468987 15266 log.cc:1196] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/9a69442abbed436c8e0b71b34012bbbd
I20260806 12:29:25.469261 15266 ts_tablet_manager.cc:1939] T 9a69442abbed436c8e0b71b34012bbbd P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting consensus metadata
I20260806 12:29:25.469794 15244 tablet_bootstrap.cc:654] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:25.469947 14691 catalog_manager.cc:4841] TS 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713): tablet 9a69442abbed436c8e0b71b34012bbbd (table test-table [id=9482e927e4914db295ff0e1d4c831836]) successfully deleted
I20260806 12:29:25.471524 15244 tablet_bootstrap.cc:492] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: No bootstrap required, opened a new log
I20260806 12:29:25.471596 15244 ts_tablet_manager.cc:1397] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 12:29:25.471979 15244 raft_consensus.cc:348] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.472110 15244 raft_consensus.cc:374] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:25.472177 15244 raft_consensus.cc:729] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Initialized, Role: FOLLOWER
I20260806 12:29:25.472337 15244 consensus_queue.cc:260] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.472636 15244 ts_tablet_manager.cc:1428] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:29:25.633916 15273 raft_consensus.cc:482] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:25.634052 15273 raft_consensus.cc:504] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.634562 15273 leader_election.cc:288] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:25.634979 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b47dfa331f4a4b9fb105aca3d06c9c" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a" is_pre_election: true
I20260806 12:29:25.634989 14872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b47dfa331f4a4b9fb105aca3d06c9c" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" is_pre_election: true
I20260806 12:29:25.635138 15165 raft_consensus.cc:2454] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 0.
I20260806 12:29:25.635180 14872 raft_consensus.cc:2454] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 0.
I20260806 12:29:25.635445 14949 leader_election.cc:302] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a52ab78a7f0d45dbbf9291061c0cba76, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:25.635644 15273 raft_consensus.cc:2781] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:25.635710 15273 raft_consensus.cc:482] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:25.635762 15273 raft_consensus.cc:3037] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:25.636785 15273 raft_consensus.cc:504] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.637127 15273 leader_election.cc:288] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 election: Requested vote from peers 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713), d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313)
I20260806 12:29:25.637468 15165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b47dfa331f4a4b9fb105aca3d06c9c" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37f7c11d091484485d10c4221cd9a7a"
I20260806 12:29:25.637413 14872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b47dfa331f4a4b9fb105aca3d06c9c" candidate_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160f9bf01dc45d4bdfe9447b3519d70"
I20260806 12:29:25.637588 15165 raft_consensus.cc:3037] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:25.637622 14872 raft_consensus.cc:3037] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:25.638672 15165 raft_consensus.cc:2454] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 1.
I20260806 12:29:25.638787 14872 raft_consensus.cc:2454] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a52ab78a7f0d45dbbf9291061c0cba76 in term 1.
I20260806 12:29:25.639046 14949 leader_election.cc:302] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a52ab78a7f0d45dbbf9291061c0cba76, d37f7c11d091484485d10c4221cd9a7a; no voters:
I20260806 12:29:25.639279 15273 raft_consensus.cc:2781] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:25.639464 15273 raft_consensus.cc:686] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 LEADER]: Becoming Leader. State: Replica: a52ab78a7f0d45dbbf9291061c0cba76, State: Running, Role: LEADER
I20260806 12:29:25.639634 15273 consensus_queue.cc:237] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } }
I20260806 12:29:25.641177 14703 catalog_manager.cc:5495] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 reported cstate change: term changed from 0 to 1, leader changed from <none> to a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a52ab78a7f0d45dbbf9291061c0cba76" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:25.663959 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.661140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.6.152.130:37111)
0806 12:29:25.661358 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:25.661362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:25.661374 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:29:25.661409 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:25.661411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:25.661449 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:25.661519 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:25.661725 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.662219 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.662971 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:25.663216 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:25.663291 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:25.663528 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:25.663600 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:25.663654 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:25.663733 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 12:29:25.663776 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260806 12:29:25.669727 15165 raft_consensus.cc:1261] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Refusing update from remote peer a52ab78a7f0d45dbbf9291061c0cba76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.669799 14872 raft_consensus.cc:1261] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Refusing update from remote peer a52ab78a7f0d45dbbf9291061c0cba76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:25.670097 15324 consensus_queue.cc:1035] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37f7c11d091484485d10c4221cd9a7a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.670243 15273 consensus_queue.cc:1035] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160f9bf01dc45d4bdfe9447b3519d70" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:25.700773 14704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:25.700918 14704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:25.702729 14704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69fee10a67c8497d87a0dd9e9258c0bf: Sending DeleteTablet for 3 replicas of tablet 37b47dfa331f4a4b9fb105aca3d06c9c
I20260806 12:29:25.703234 14852 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b47dfa331f4a4b9fb105aca3d06c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:38924
I20260806 12:29:25.703262 14999 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b47dfa331f4a4b9fb105aca3d06c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:40422
I20260806 12:29:25.703429 15266 tablet_replica.cc:330] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: stopping tablet replica
I20260806 12:29:25.703455 15267 tablet_replica.cc:330] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: stopping tablet replica
I20260806 12:29:25.703473 15145 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b47dfa331f4a4b9fb105aca3d06c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:25 UTC) from {username='oryx', principal='oryx/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:57648
I20260806 12:29:25.703574 15266 raft_consensus.cc:2229] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:25.703595 15267 raft_consensus.cc:2229] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:29:25.703644 15268 tablet_replica.cc:330] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: stopping tablet replica
I20260806 12:29:25.703694 15266 raft_consensus.cc:2258] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.703742 15268 raft_consensus.cc:2229] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:25.703780 15267 raft_consensus.cc:2258] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.703861 15268 raft_consensus.cc:2258] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:25.704509 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 14783
I20260806 12:29:25.704559 15266 ts_tablet_manager.cc:1905] T 37b47dfa331f4a4b9fb105aca3d06c9c P 1160f9bf01dc45d4bdfe9447b3519d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.704556 15267 ts_tablet_manager.cc:1905] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.704671 15268 ts_tablet_manager.cc:1905] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:25.707072 15267 ts_tablet_manager.cc:1918] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:29:25.707194 15267 log.cc:1196] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/37b47dfa331f4a4b9fb105aca3d06c9c
I20260806 12:29:25.707072 15268 ts_tablet_manager.cc:1918] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 12:29:25.707310 15268 log.cc:1196] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/37b47dfa331f4a4b9fb105aca3d06c9c
I20260806 12:29:25.707612 15268 ts_tablet_manager.cc:1939] T 37b47dfa331f4a4b9fb105aca3d06c9c P d37f7c11d091484485d10c4221cd9a7a: Deleting consensus metadata
I20260806 12:29:25.708308 14689 catalog_manager.cc:4841] TS d37f7c11d091484485d10c4221cd9a7a (127.6.152.131:45313): tablet 37b47dfa331f4a4b9fb105aca3d06c9c (table test-table [id=e8209641252d43f98832027fb7e65460]) successfully deleted
I20260806 12:29:25.708432 15267 ts_tablet_manager.cc:1939] T 37b47dfa331f4a4b9fb105aca3d06c9c P a52ab78a7f0d45dbbf9291061c0cba76: Deleting consensus metadata
I20260806 12:29:25.709085 14688 catalog_manager.cc:4841] TS a52ab78a7f0d45dbbf9291061c0cba76 (127.6.152.130:37111): tablet 37b47dfa331f4a4b9fb105aca3d06c9c (table test-table [id=e8209641252d43f98832027fb7e65460]) successfully deleted
W20260806 12:29:25.715188 14691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:39713 (error 108)
I20260806 12:29:25.715252 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 14931
I20260806 12:29:25.717226 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:25.716736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39713 (local address 127.0.0.1:38944)
0806 12:29:25.717008 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:25.717094 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":116,"threads_started":1}
W20260806 12:29:25.717352 14691 catalog_manager.cc:4584] TS 1160f9bf01dc45d4bdfe9447b3519d70 (127.6.152.129:39713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37b47dfa331f4a4b9fb105aca3d06c9c: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39713: connect: Connection refused (error 111)
I20260806 12:29:25.723398 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15077
I20260806 12:29:25.730005 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 14672
2026-08-06T12:29:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3241 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[15337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[15337](info): set up 2 sockets
Aug 06 12:29:25 dist-test-slave-1xrh krb5kdc[15337](info): commencing operation
krb5kdc: starting...
W20260806 12:29:26.791990 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:26 dist-test-slave-1xrh krb5kdc[15337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:26.917471 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39067
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:39067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:27.025902 15351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:27.026144 15351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:27.026182 15351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:27.030088 15351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:27.030143 15351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:27.030162 15351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:27.030179 15351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:27.030193 15351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:27.035175 15351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:39067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:27.036114 15351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:27.037051 15351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:27.045946 15356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:27.045990 15357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:27.046056 15351 server_base.cc:1034] running on GCE node
W20260806 12:29:27.045964 15359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:27.046799 15351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:27.047909 15351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:27.049077 15351 hybrid_clock.cc:648] HybridClock initialized: now 1786019367049059 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T12:29:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1333 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:27 dist-test-slave-1xrh krb5kdc[15364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:27 dist-test-slave-1xrh krb5kdc[15364](info): set up 2 sockets
Aug 06 12:29:27 dist-test-slave-1xrh krb5kdc[15364](info): commencing operation
krb5kdc: starting...
W20260806 12:29:28.195884 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:28.322024 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33591
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:33591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:28.434643 15377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:28.434902 15377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:28.434942 15377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:28.439000 15377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:28.439055 15377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:28.439074 15377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:28.439090 15377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:28.439105 15377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:28.444310 15377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:33591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:28.445248 15377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:28.446208 15377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:28.454627 15385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:28.454658 15383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:28.454653 15382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:28.454803 15377 server_base.cc:1034] running on GCE node
I20260806 12:29:28.455554 15377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:28.456584 15377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:28.457772 15377 hybrid_clock.cc:648] HybridClock initialized: now 1786019368457758 us; error 34 us; skew 500 ppm
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:28.460661 15377 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:28.461494 15377 webserver.cc:460] Webserver started at http://127.6.152.190:37799/ using document root <none> and password file <none>
I20260806 12:29:28.462031 15377 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:28.462123 15377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:28.462358 15377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:28.464387 15377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "9a557272ba284d889d6c4c48966e8326"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "17687c8cd598e3555a23c3c1244dd10c"
server_key_iv: "9906dde74482f67ebdfdc751395b8f82"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.464952 15377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "9a557272ba284d889d6c4c48966e8326"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "17687c8cd598e3555a23c3c1244dd10c"
server_key_iv: "9906dde74482f67ebdfdc751395b8f82"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.468472 15377 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:28.470698 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.471927 15377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:28.472074 15377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "9a557272ba284d889d6c4c48966e8326"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "17687c8cd598e3555a23c3c1244dd10c"
server_key_iv: "9906dde74482f67ebdfdc751395b8f82"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.472180 15377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:28.497066 15377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:28.500749 15377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:28.501034 15377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:28.508103 15377 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:33591
I20260806 12:29:28.508174 15449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:33591 every 8 connection(s)
I20260806 12:29:28.509053 15377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:28.511739 15450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:28.517061 15450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Bootstrap starting.
I20260806 12:29:28.518007 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15377
I20260806 12:29:28.518146 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:28.518448 6754 external_mini_cluster.cc:1428] Setting key 3d4256a6ffb2c97f7009e9eb0e67fb26
I20260806 12:29:28.519814 15450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:28.520576 15450 log.cc:824] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:28.522686 15450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: No bootstrap required, opened a new log
I20260806 12:29:28.527235 15450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:28.527463 15450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:28.527542 15450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a557272ba284d889d6c4c48966e8326, State: Initialized, Role: FOLLOWER
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:28.528003 15450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:28.528159 15450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:28.528232 15450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:28.528339 15450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:28.529536 15450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:28.529896 15450 leader_election.cc:302] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a557272ba284d889d6c4c48966e8326; no voters:
I20260806 12:29:28.530234 15450 leader_election.cc:288] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:28.530369 15456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:28.530594 15456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 LEADER]: Becoming Leader. State: Replica: 9a557272ba284d889d6c4c48966e8326, State: Running, Role: LEADER
I20260806 12:29:28.530889 15456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:28.531620 15450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:28.532610 15457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a557272ba284d889d6c4c48966e8326. Latest consensus state: current_term: 1 leader_uuid: "9a557272ba284d889d6c4c48966e8326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } } }
I20260806 12:29:28.532755 15457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:28.533113 15458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a557272ba284d889d6c4c48966e8326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } } }
I20260806 12:29:28.533215 15458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:28.533515 15465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:28.535003 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:28.519666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.6.152.190:33591)
0806 12:29:28.520143 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:28.520159 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:28.520195 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:29:28.520862 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:28.520880 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:28.521187 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:28.521455 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:28.522374 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.523466 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.524721 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.525189 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.525313 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:28.528318 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:28.528325 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:28.528342 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:28.528377 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:28.530403 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:28.530953 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:28.530958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:28.530964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:28.531098 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:28.531519 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:28.531522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:28.531524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:28.532063 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:28.533016 (+ 953us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:28.533178 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 12:29:28.533519 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":159,"threads_started":1}
I20260806 12:29:28.536906 15465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:28.541908 15465 catalog_manager.cc:1347] Generated new cluster ID: 9999cd517cd14d7ca077620fef586aa0
I20260806 12:29:28.541967 15465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:28.550791 15465 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:28.551674 15465 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:28.563465 15465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Generated new TSK 0
I20260806 12:29:28.564170 15465 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:29:28.598140 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33591
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:28.709249 15478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:28.709641 15478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:28.709688 15478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:28.713776 15478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:28.713828 15478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:28.713884 15478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:28.718986 15478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:28.720002 15478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:28.720952 15478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:28.730260 15484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:28.730237 15483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:28.730237 15486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:28.730281 15478 server_base.cc:1034] running on GCE node
I20260806 12:29:28.730858 15478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:28.731559 15478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:28.732710 15478 hybrid_clock.cc:648] HybridClock initialized: now 1786019368732683 us; error 41 us; skew 500 ppm
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:28.735800 15478 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:29:28.736644 15478 webserver.cc:460] Webserver started at http://127.6.152.129:34255/ using document root <none> and password file <none>
I20260806 12:29:28.737174 15478 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:28.737263 15478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:28.737490 15478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:28.739480 15478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "c503bf2e04aa48c2ae4f6c4868a7ef83"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "51268321c30d0b163f59835dbaedc4f2"
server_key_iv: "ff8ec247221fd81ab2a5a3de0e7391b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.740047 15478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "c503bf2e04aa48c2ae4f6c4868a7ef83"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "51268321c30d0b163f59835dbaedc4f2"
server_key_iv: "ff8ec247221fd81ab2a5a3de0e7391b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.743637 15478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:29:28.745862 15494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.746929 15478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:28.747059 15478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "c503bf2e04aa48c2ae4f6c4868a7ef83"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "51268321c30d0b163f59835dbaedc4f2"
server_key_iv: "ff8ec247221fd81ab2a5a3de0e7391b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.747215 15478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:28.758304 15478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:28.760859 15478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:28.761059 15478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:28.761587 15478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:28.762343 15478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:28.762415 15478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.762475 15478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:28.762522 15478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.771178 15478 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:45327
I20260806 12:29:28.771214 15615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:45327 every 8 connection(s)
I20260806 12:29:28.772001 15478 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:28.774326 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15478
I20260806 12:29:28.774490 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:28.774823 6754 external_mini_cluster.cc:1428] Setting key 7b0ca90be927213c1573a97790c7eed8
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:29:28.785964 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:28.773749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:45721 (local address 127.6.152.190:33591)
0806 12:29:28.773881 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:28.773885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:28.775003 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 12:29:28.776492 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:28.776495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:28.776550 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:28.776675 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:28.777800 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.778340 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.779537 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.779772 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.779885 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:28.782313 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:28.782316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:28.782318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:28.782341 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:28.783842 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:28.784378 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:28.784382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:28.784384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:28.784468 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:28.784865 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:28.784869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:28.784870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:28.785074 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:28.785183 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:28.785661 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 12:29:28.785786 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:29:28.786880 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:28.774124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:45721)
0806 12:29:28.774762 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:28.774861 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:28.776241 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:28.776850 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:28.776857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:28.777191 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:28.777643 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:28.777664 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.778479 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.778483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:28.779411 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:28.779421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.779923 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.779927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:28.780112 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:28.780699 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:28.780715 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:28.782132 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:28.783968 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:28.783974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:28.783988 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:28.784270 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:28.784583 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:28.784586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:28.784588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:28.784759 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:28.785194 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:28.785202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:28.785427 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 12:29:28.785616 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 12:29:28.785920 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":160,"threads_started":1}
I20260806 12:29:28.788285 15616 heartbeater.cc:347] Connected to a master server at 127.6.152.190:33591
I20260806 12:29:28.788539 15616 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:28.789028 15616 heartbeater.cc:511] Master 127.6.152.190:33591 requested a full tablet report, sending...
I20260806 12:29:28.790617 15411 ts_manager.cc:194] Registered new tserver with Master: c503bf2e04aa48c2ae4f6c4868a7ef83 (127.6.152.129:45327)
I20260806 12:29:28.792405 15411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:45721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:28.802126 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33591
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:28.909299 15621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:28.909545 15621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:28.909586 15621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:28.913367 15621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:28.913416 15621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:28.913473 15621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:28.918332 15621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:28.919236 15621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:28.920120 15621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:28.929265 15627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:28.929299 15628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:28.929278 15630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:28.929581 15621 server_base.cc:1034] running on GCE node
I20260806 12:29:28.930006 15621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:28.930636 15621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:28.931814 15621 hybrid_clock.cc:648] HybridClock initialized: now 1786019368931783 us; error 54 us; skew 500 ppm
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:28.934477 15621 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:28.935374 15621 webserver.cc:460] Webserver started at http://127.6.152.130:36309/ using document root <none> and password file <none>
I20260806 12:29:28.935941 15621 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:28.936033 15621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:28.936257 15621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:28.938166 15621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "107f61fa6f044c94960fdd64380a544d"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "02a0518a6099f956c89afd485a2e8cf6"
server_key_iv: "9a45b2dd79e89d670d23f76f1f1e5e13"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.938727 15621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "107f61fa6f044c94960fdd64380a544d"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "02a0518a6099f956c89afd485a2e8cf6"
server_key_iv: "9a45b2dd79e89d670d23f76f1f1e5e13"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.942214 15621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:29:28.944584 15637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.945678 15621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:28.945809 15621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "107f61fa6f044c94960fdd64380a544d"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "02a0518a6099f956c89afd485a2e8cf6"
server_key_iv: "9a45b2dd79e89d670d23f76f1f1e5e13"
server_key_version: "encryptionkey@0"
I20260806 12:29:28.945914 15621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:28.973348 15621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:28.976244 15621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:28.976472 15621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:28.977092 15621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:28.977905 15621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:28.977986 15621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.978060 15621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:28.978094 15621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:28.987167 15621 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:34879
I20260806 12:29:28.987236 15753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:34879 every 8 connection(s)
I20260806 12:29:28.988085 15621 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:29:28 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:28.997941 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15621
I20260806 12:29:28.998091 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:28.998467 6754 external_mini_cluster.cc:1428] Setting key 288a7ba04ab3d37ce2b0d7627004a6dc
I20260806 12:29:29.002355 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:28.989943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:35507 (local address 127.6.152.190:33591)
0806 12:29:28.990097 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:28.990101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:28.990902 (+ 801us) server_negotiation.cc:407] Connection header received
0806 12:29:28.992267 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:28.992270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:28.992332 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:28.992418 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:28.993592 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.994135 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.995381 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.995611 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.995725 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:28.998140 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:28.998147 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:28.998152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:28.998186 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:29.000009 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:29.000680 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:29.000682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:29.000683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:29.000760 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:29.001172 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:29.001174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:29.001175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:29.001343 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:29.001420 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:29.002029 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 12:29:29.002122 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 12:29:29.003350 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:28.990288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:35507)
0806 12:29:28.990745 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:28.990787 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:28.992015 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:28.992611 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:28.992618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:28.992968 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:28.993430 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:28.993442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.994273 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.994277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:28.995252 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:28.995262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:28.995769 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:28.995773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:28.995939 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:28.996464 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:28.996480 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:28.997931 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:29.000191 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:29.000198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:29.000212 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:29.000563 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:29.000855 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:29.000861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:29.000866 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:29.001074 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:29.001444 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:29.001452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:29.001766 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 12:29:29.001995 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 12:29:29.002365 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260806 12:29:29.004602 15754 heartbeater.cc:347] Connected to a master server at 127.6.152.190:33591
I20260806 12:29:29.004853 15754 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:29.005342 15754 heartbeater.cc:511] Master 127.6.152.190:33591 requested a full tablet report, sending...
I20260806 12:29:29.006496 15411 ts_manager.cc:194] Registered new tserver with Master: 107f61fa6f044c94960fdd64380a544d (127.6.152.130:34879)
I20260806 12:29:29.007426 15411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:35507
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:29.028323 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33591
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:29.137187 15759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:29.137446 15759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:29.137523 15759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:29.141391 15759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:29.141438 15759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:29.141492 15759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:29.146476 15759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:29.147440 15759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:29.148418 15759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:29.157979 15765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.158041 15759 server_base.cc:1034] running on GCE node
W20260806 12:29:29.158063 15767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:29.157979 15764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.158600 15759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:29.159263 15759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:29.160437 15759 hybrid_clock.cc:648] HybridClock initialized: now 1786019369160426 us; error 31 us; skew 500 ppm
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:29.163090 15759 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:29.163967 15759 webserver.cc:460] Webserver started at http://127.6.152.131:33083/ using document root <none> and password file <none>
I20260806 12:29:29.164448 15759 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:29.164505 15759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:29.164659 15759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:29.166518 15759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "530c45980ee14653b264e28a9878870a"
format_stamp: "Formatted at 2026-08-06 12:29:29 on dist-test-slave-1xrh"
server_key: "438007075c9691619fcbefdef674276e"
server_key_iv: "f8f7f1c9cb122376a4b2630d83f63a72"
server_key_version: "encryptionkey@0"
I20260806 12:29:29.167038 15759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "530c45980ee14653b264e28a9878870a"
format_stamp: "Formatted at 2026-08-06 12:29:29 on dist-test-slave-1xrh"
server_key: "438007075c9691619fcbefdef674276e"
server_key_iv: "f8f7f1c9cb122376a4b2630d83f63a72"
server_key_version: "encryptionkey@0"
I20260806 12:29:29.170521 15759 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:29:29.172863 15775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.174047 15759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 12:29:29.174171 15759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "530c45980ee14653b264e28a9878870a"
format_stamp: "Formatted at 2026-08-06 12:29:29 on dist-test-slave-1xrh"
server_key: "438007075c9691619fcbefdef674276e"
server_key_iv: "f8f7f1c9cb122376a4b2630d83f63a72"
server_key_version: "encryptionkey@0"
I20260806 12:29:29.174279 15759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:29.193060 15759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:29.195866 15759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:29.196066 15759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:29.196630 15759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:29.197423 15759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:29.197470 15759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.197536 15759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:29.197575 15759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.207039 15759 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35565
I20260806 12:29:29.207064 15890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35565 every 8 connection(s)
I20260806 12:29:29.207894 15759 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:29.214313 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15759
I20260806 12:29:29.214453 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:29.214764 6754 external_mini_cluster.cc:1428] Setting key 69aa2d2d76bcbb4bb5e1c5f4dc5e0d44
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:29.221365 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.209651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:41705 (local address 127.6.152.190:33591)
0806 12:29:29.209781 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.209792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.210545 (+ 753us) server_negotiation.cc:407] Connection header received
0806 12:29:29.211887 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.211891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.211959 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.212080 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.213211 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.213765 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.215374 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.215700 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.215819 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.218308 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:29.218311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:29.218313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:29.218339 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:29.219638 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:29.220085 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:29.220088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:29.220089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:29.220160 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:29.220447 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:29.220449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:29.220450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:29.220621 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:29.220716 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:29.221143 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 12:29:29.221243 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:29:29.222110 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.209943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:41705)
0806 12:29:29.210376 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.210416 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.211670 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.212251 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.212258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.212598 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.213056 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.213069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.213901 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.213909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.215246 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.215256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.215874 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.215877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.216078 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.216672 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.216687 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.218158 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:29.219738 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:29.219742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:29.219753 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:29.219998 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:29.220237 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:29.220239 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:29.220240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:29.220369 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:29.220731 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:29.220735 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:29.220945 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 12:29:29.221092 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 12:29:29.221317 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":118,"threads_started":1}
I20260806 12:29:29.223285 15891 heartbeater.cc:347] Connected to a master server at 127.6.152.190:33591
I20260806 12:29:29.223486 15891 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:29.223922 15891 heartbeater.cc:511] Master 127.6.152.190:33591 requested a full tablet report, sending...
I20260806 12:29:29.224867 15411 ts_manager.cc:194] Registered new tserver with Master: 530c45980ee14653b264e28a9878870a (127.6.152.131:35565)
I20260806 12:29:29.225436 15411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:41705
I20260806 12:29:29.228454 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab.
Entry for principal oryx/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab.
WARNING: no policy specified for oryx/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab.
Entry for principal oryx/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab.
WARNING: no policy specified for oryx/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab.
Entry for principal oryx/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab.
I20260806 12:29:29.301122 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15478
I20260806 12:29:29.308758 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15621
I20260806 12:29:29.317251 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15759
I20260806 12:29:29.326881 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15377
I20260806 12:29:29.333391 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33591
--webserver_interface=127.6.152.190
--webserver_port=37799
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:33591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:29.443781 15901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:29.444041 15901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:29.444079 15901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:29.447873 15901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:29.447919 15901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:29.447938 15901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:29.447953 15901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:29.447968 15901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:29.453068 15901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:33591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=37799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:29.454027 15901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:29.455089 15901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:29.463222 15906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:29.463238 15907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.463403 15901 server_base.cc:1034] running on GCE node
W20260806 12:29:29.463246 15909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.463960 15901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:29.464938 15901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:29.466096 15901 hybrid_clock.cc:648] HybridClock initialized: now 1786019369466083 us; error 23 us; skew 500 ppm
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:29.468869 15901 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:29.469704 15901 webserver.cc:460] Webserver started at http://127.6.152.190:37799/ using document root <none> and password file <none>
I20260806 12:29:29.470196 15901 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:29.470266 15901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:29.473898 15901 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:29.475900 15917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.477037 15901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:29.477144 15901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "9a557272ba284d889d6c4c48966e8326"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "17687c8cd598e3555a23c3c1244dd10c"
server_key_iv: "9906dde74482f67ebdfdc751395b8f82"
server_key_version: "encryptionkey@0"
I20260806 12:29:29.477629 15901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:29.490113 15901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:29.492987 15901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:29.493135 15901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:29.500105 15901 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:33591
I20260806 12:29:29.500120 15970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:33591 every 8 connection(s)
I20260806 12:29:29.501120 15901 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:29.505368 15971 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:29:29.506930 15971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Bootstrap starting.
I20260806 12:29:29.509943 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15901
I20260806 12:29:29.510427 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:45327
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:33591
--webserver_port=34255
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:29:29.523878 15971 log.cc:824] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:29.530089 15971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:29:29.530452 15971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Bootstrap complete.
I20260806 12:29:29.534634 15971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:29.535058 15971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a557272ba284d889d6c4c48966e8326, State: Initialized, Role: FOLLOWER
I20260806 12:29:29.535595 15971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:29.535741 15971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:29.535830 15971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:29.535943 15971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:29:29.537794 15971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:29.538117 15971 leader_election.cc:302] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a557272ba284d889d6c4c48966e8326; no voters:
I20260806 12:29:29.538448 15971 leader_election.cc:288] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:29:29.538532 15975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:29:29.538705 15975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [term 2 LEADER]: Becoming Leader. State: Replica: 9a557272ba284d889d6c4c48966e8326, State: Running, Role: LEADER
I20260806 12:29:29.538991 15975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } }
I20260806 12:29:29.539640 15971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:29.540453 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a557272ba284d889d6c4c48966e8326. Latest consensus state: current_term: 2 leader_uuid: "9a557272ba284d889d6c4c48966e8326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } } }
I20260806 12:29:29.540462 15976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9a557272ba284d889d6c4c48966e8326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a557272ba284d889d6c4c48966e8326" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33591 } } }
I20260806 12:29:29.540585 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:29.540623 15976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:29.540999 15981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:29.544166 15981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:29.544848 15981 catalog_manager.cc:1259] Loaded cluster ID: 9999cd517cd14d7ca077620fef586aa0
I20260806 12:29:29.544927 15981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:29.546548 15981 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:29.547115 15981 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a557272ba284d889d6c4c48966e8326: Loaded TSK: 0
I20260806 12:29:29.548061 15981 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:29:29.629472 15973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:29.629746 15973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:29.629824 15973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:29.633879 15973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:29.633970 15973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:29.634058 15973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:29.639104 15973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.129.keytab
--principal=oryx/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:45327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=34255
--tserver_master_addrs=127.6.152.190:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:29.640137 15973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:29.641233 15973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:29.651674 15997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.651705 15973 server_base.cc:1034] running on GCE node
W20260806 12:29:29.651779 15998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:29.651973 16000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.652410 15973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:29.653187 15973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:29.654412 15973 hybrid_clock.cc:648] HybridClock initialized: now 1786019369654372 us; error 47 us; skew 500 ppm
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019369, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:29.657975 15973 init.cc:377] Logged in from keytab as oryx/127.6.152.129@KRBTEST.COM (short username oryx)
I20260806 12:29:29.659082 15973 webserver.cc:460] Webserver started at http://127.6.152.129:34255/ using document root <none> and password file <none>
I20260806 12:29:29.659811 15973 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:29.659927 15973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:29.664947 15973 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:29.667564 16007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.669054 15973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:29:29.669178 15973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "c503bf2e04aa48c2ae4f6c4868a7ef83"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "51268321c30d0b163f59835dbaedc4f2"
server_key_iv: "ff8ec247221fd81ab2a5a3de0e7391b7"
server_key_version: "encryptionkey@0"
I20260806 12:29:29.669853 15973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:29.696581 15973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:29.699949 15973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:29.700145 15973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:29.700768 15973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:29.701709 15973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:29.701768 15973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.701819 15973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:29.701848 15973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.711966 15973 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:45327
I20260806 12:29:29.712009 16124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:45327 every 8 connection(s)
I20260806 12:29:29.712949 15973 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:29.717428 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 15973
I20260806 12:29:29.717976 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:34879
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:33591
--webserver_port=36309
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:29.726410 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.715082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:42765)
0806 12:29:29.715611 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.715651 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.716833 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.718137 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.718148 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.718538 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.718972 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.718986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.720464 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.720470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.721452 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.721462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.722572 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.722575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.722705 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.723370 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.723386 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.725732 (+ 2346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":138,"threads_started":1}
W20260806 12:29:29.726974 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:29.714956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:42765 (local address 127.6.152.190:33591)
0806 12:29:29.715479 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.715500 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.715764 (+ 264us) server_negotiation.cc:407] Connection header received
0806 12:29:29.717267 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.717284 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.717685 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.718041 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.719183 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.720313 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.721588 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.722407 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.722606 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.726045 (+ 3439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:42765: BlockingRecv error: recv got EOF from 127.6.152.129:42765 (error 108)
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":154,"threads_started":1}
W20260806 12:29:29.727005 16125 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:33591 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.152.190:33591: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:29.732385 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.727559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:55671)
0806 12:29:29.727650 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.727666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.727764 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.728000 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.728003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.728175 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.728259 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.728264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.728964 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.728969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.729486 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.729493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.729942 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.729946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.730041 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.730507 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.730517 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.732254 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 12:29:29.732717 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:29.727653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:55671 (local address 127.6.152.190:33591)
0806 12:29:29.727786 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.727804 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.727820 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:29:29.727855 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.727858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.727902 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.727986 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.728373 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.728855 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.729615 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.729831 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.729940 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.732357 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:55671: BlockingRecv error: recv got EOF from 127.6.152.129:55671 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:29.737512 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.732862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:45983)
0806 12:29:29.733027 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.733042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.733112 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.733333 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.733336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.733480 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.733552 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.733557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.734244 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.734250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.734744 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.734751 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.735239 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.735243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.735327 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.735726 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.735736 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.737402 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 12:29:29.737633 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:29.732967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:45983 (local address 127.6.152.190:33591)
0806 12:29:29.733104 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.733108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.733122 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:29.733189 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.733192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.733228 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.733318 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.733662 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.734135 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.734862 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.735094 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.735236 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.737518 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:45983: BlockingRecv error: recv got EOF from 127.6.152.129:45983 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 12:29:29.737686 16125 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:29:29.830708 16129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:29.830979 16129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:29.831072 16129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:29.835084 16129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:29.835176 16129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:29.835289 16129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:29.840492 16129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.130.keytab
--principal=oryx/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:34879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=36309
--tserver_master_addrs=127.6.152.190:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:29.841553 16129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:29.842530 16129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:29.851929 16137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:29.852041 16135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:29.851997 16134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:29.852299 16129 server_base.cc:1034] running on GCE node
I20260806 12:29:29.852679 16129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:29.853340 16129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:29.854501 16129 hybrid_clock.cc:648] HybridClock initialized: now 1786019369854486 us; error 24 us; skew 500 ppm
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019369, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:29.857288 16129 init.cc:377] Logged in from keytab as oryx/127.6.152.130@KRBTEST.COM (short username oryx)
I20260806 12:29:29.858057 16129 webserver.cc:460] Webserver started at http://127.6.152.130:36309/ using document root <none> and password file <none>
I20260806 12:29:29.858573 16129 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:29.858682 16129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:29.862165 16129 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:29.864096 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.865263 16129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:29.865396 16129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "107f61fa6f044c94960fdd64380a544d"
format_stamp: "Formatted at 2026-08-06 12:29:28 on dist-test-slave-1xrh"
server_key: "02a0518a6099f956c89afd485a2e8cf6"
server_key_iv: "9a45b2dd79e89d670d23f76f1f1e5e13"
server_key_version: "encryptionkey@0"
I20260806 12:29:29.865900 16129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:29.886868 16129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:29.889729 16129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:29.889926 16129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:29.890472 16129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:29.891342 16129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:29.891412 16129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.891503 16129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:29.891556 16129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:29.900636 16129 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:34879
I20260806 12:29:29.900671 16266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:34879 every 8 connection(s)
I20260806 12:29:29.901612 16129 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:29.905066 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16129
I20260806 12:29:29.905601 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:35565
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:33591
--webserver_port=33083
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:29:29.914202 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:29.903521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:53899 (local address 127.6.152.190:33591)
0806 12:29:29.903655 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.903659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.904424 (+ 765us) server_negotiation.cc:407] Connection header received
0806 12:29:29.906280 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.906284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.906337 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.906404 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.907966 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.908812 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.909918 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.910360 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.910484 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.914101 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:53899: BlockingRecv error: recv got EOF from 127.6.152.130:53899 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:29:29.915082 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.903843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:53899)
0806 12:29:29.904269 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.904309 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.906016 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.906637 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.906649 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.907217 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.907793 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.907806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.908949 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.908951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.909781 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.909790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.910525 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.910528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.910670 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.911319 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.911341 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.913987 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
W20260806 12:29:29.915617 16267 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:33591 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.152.190:33591: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:29.921157 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.916090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:40061)
0806 12:29:29.916183 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.916197 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.916294 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.916565 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.916568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.916719 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.916799 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.916804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.917531 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.917536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.918079 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.918086 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.918537 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.918541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.918630 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.919251 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.919263 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.921045 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 12:29:29.921556 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:29.916207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:40061 (local address 127.6.152.190:33591)
0806 12:29:29.916364 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.916369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.916381 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:29:29.916414 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.916418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.916456 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.916551 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.916913 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.917424 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.918201 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.918425 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.918538 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.921178 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:40061: BlockingRecv error: recv got EOF from 127.6.152.130:40061 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:29 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:29.926815 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:29.921668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:49637)
0806 12:29:29.921882 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:29.921894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:29.921987 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:29.922190 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:29.922193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:29.922349 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:29.922431 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.922436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.923087 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.923093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.924035 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:29.924046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.924446 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.924450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:29.924531 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.924929 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:29.924942 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:29.926686 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155}
W20260806 12:29:29.926903 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:29.921875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:49637 (local address 127.6.152.190:33591)
0806 12:29:29.922002 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:29.922005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:29.922018 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:29.922053 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:29.922055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:29.922087 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:29.922152 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:29.922540 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.922988 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.924146 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:29.924343 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:29.924420 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:29.926793 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:49637: BlockingRecv error: recv got EOF from 127.6.152.130:49637 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 12:29:29.927191 16267 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:29:30.019094 16270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:30.019438 16270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:30.019528 16270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:30.023545 16270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:30.023619 16270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:30.023702 16270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:30.029006 16270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/krb5kdc/oryx_127.6.152.131.keytab
--principal=oryx/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:35565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=33083
--tserver_master_addrs=127.6.152.190:33591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:30.030040 16270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:30.031056 16270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:30.040745 16279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:30.040729 16277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:30.040733 16275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:30.041262 16270 server_base.cc:1034] running on GCE node
I20260806 12:29:30.041656 16270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:30.042382 16270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:30.043565 16270 hybrid_clock.cc:648] HybridClock initialized: now 1786019370043540 us; error 48 us; skew 500 ppm
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019370, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:30.046278 16270 init.cc:377] Logged in from keytab as oryx/127.6.152.131@KRBTEST.COM (short username oryx)
I20260806 12:29:30.047237 16270 webserver.cc:460] Webserver started at http://127.6.152.131:33083/ using document root <none> and password file <none>
I20260806 12:29:30.047808 16270 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:30.047919 16270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:30.051693 16270 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 12:29:30.053627 16286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:30.054793 16270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:30.054921 16270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "530c45980ee14653b264e28a9878870a"
format_stamp: "Formatted at 2026-08-06 12:29:29 on dist-test-slave-1xrh"
server_key: "438007075c9691619fcbefdef674276e"
server_key_iv: "f8f7f1c9cb122376a4b2630d83f63a72"
server_key_version: "encryptionkey@0"
I20260806 12:29:30.055559 16270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:30.075773 16270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:30.078387 16270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:30.078591 16270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:30.079205 16270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:30.079993 16270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:30.080063 16270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:30.080128 16270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:30.080181 16270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:30.089227 16270 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35565
I20260806 12:29:30.089250 16404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35565 every 8 connection(s)
I20260806 12:29:30.090183 16270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:30.092909 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16270
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:29:30.102501 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:30.091997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:35103 (local address 127.6.152.190:33591)
0806 12:29:30.092171 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:30.092177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:30.093032 (+ 855us) server_negotiation.cc:407] Connection header received
0806 12:29:30.094641 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:30.094645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:30.094691 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:30.094769 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:30.096136 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.096925 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.098414 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.098819 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.098930 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.102380 (+ 3450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:35103: BlockingRecv error: recv got EOF from 127.6.152.131:35103 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 12:29:30.103295 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:30.092403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:35103)
0806 12:29:30.092871 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:30.092916 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:30.094438 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:30.094934 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:30.094941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:30.095370 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:30.095960 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.095977 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.097055 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.097058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.098297 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.098307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.098973 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.098976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.099110 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.099736 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:30.099751 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:30.102281 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":160,"threads_started":1}
I20260806 12:29:30.103591 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:30.093994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51062 (local address 127.6.152.190:33591)
0806 12:29:30.094248 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:30.094251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:30.094262 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:29:30.094304 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:30.094306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:30.094345 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:30.094449 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:30.095239 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.096018 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.097209 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.097544 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.097657 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.099295 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:30.099303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:30.099321 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:30.099356 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:30.101125 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:30.101609 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:30.101614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:30.101618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:30.101732 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:30.102024 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:30.102029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:30.102031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:30.102346 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:30.102554 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:30.103229 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 12:29:30.103464 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
W20260806 12:29:30.103734 16405 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:33591 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.152.190:33591: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:30.108893 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:30.104161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:39889)
0806 12:29:30.104271 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:30.104286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:30.104360 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:30.104589 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:30.104592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:30.104746 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:30.104852 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.104858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.105668 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.105672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.106383 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.106396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.106789 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.106792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.106857 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.107333 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:30.107345 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:30.108813 (+ 1468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 12:29:30.109069 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:30.104246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:39889 (local address 127.6.152.190:33591)
0806 12:29:30.104348 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:30.104352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:30.104366 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:30.104430 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:30.104433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:30.104471 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:30.104555 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:30.104962 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.105547 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.106497 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.106693 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.106770 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.108952 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:39889: BlockingRecv error: recv got EOF from 127.6.152.131:39889 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:30.114069 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:30.109328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:60943)
0806 12:29:30.109434 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:30.109448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:30.109513 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:30.109703 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:30.109705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:30.109820 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:30.109880 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.109884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.110501 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.110504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.111212 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.111224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.111882 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.111885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.111993 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.112438 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:30.112451 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:30.113988 (+ 1537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 12:29:30.114305 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:30.109420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:60943 (local address 127.6.152.190:33591)
0806 12:29:30.109546 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:30.109549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:30.109559 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:29:30.109585 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:30.109587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:30.109615 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:30.109677 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:30.109972 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.110416 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.111364 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.111755 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.111890 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.114200 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:60943: BlockingRecv error: recv got EOF from 127.6.152.131:60943 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 12:29:30.114378 16405 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:30.744163 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:30.738391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:50041)
0806 12:29:30.738698 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:30.738717 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:30.738823 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:30.739089 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:30.739091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:30.739449 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:30.739567 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.739573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.740419 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.740422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.741047 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.741055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.741525 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.741533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.741638 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.742198 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:30.742211 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:30.744040 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":117,"threads_started":1}
W20260806 12:29:30.744308 16412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:30.738468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:50041 (local address 127.6.152.190:33591)
0806 12:29:30.738764 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:30.738770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:30.738787 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:29:30.738944 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:30.738947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:30.738993 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:30.739071 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:30.739679 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.740278 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.741150 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.741370 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.741506 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.744173 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:50041: BlockingRecv error: recv got EOF from 127.6.152.129:50041 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:30 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:30.933307 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:30.927779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:54963)
0806 12:29:30.928053 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:30.928069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:30.928170 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:30.928404 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:30.928406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:30.928582 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:30.928677 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.928682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.929387 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.929389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.929947 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:30.929953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.930337 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.930339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:30.930397 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.930944 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:30.930956 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:30.933129 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":70,"threads_started":1}
W20260806 12:29:30.933349 16412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:30.927874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:54963 (local address 127.6.152.190:33591)
0806 12:29:30.928070 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:30.928074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:30.928144 (+ 70us) server_negotiation.cc:407] Connection header received
0806 12:29:30.928250 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:30.928252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:30.928295 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:30.928364 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:30.928775 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.929295 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.930045 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:30.930244 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:30.930316 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:30.933247 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:54963: BlockingRecv error: recv got EOF from 127.6.152.130:54963 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 12:29:31 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:31 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:31.120669 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:31.114961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:45327)
0806 12:29:31.115320 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:31.115340 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:31.115434 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:31.115670 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:31.115672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:31.115849 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:31.115985 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:31.115990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.116797 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.116801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:31.117669 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:31.117683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.118158 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.118161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:31.118230 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:31.118711 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:31.118740 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:31.120563 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":219,"threads_started":1}
W20260806 12:29:31.120922 16412 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:31.115060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:45327 (local address 127.6.152.190:33591)
0806 12:29:31.115215 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:31.115219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:31.115406 (+ 187us) server_negotiation.cc:407] Connection header received
0806 12:29:31.115518 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:31.115521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:31.115569 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:31.115635 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:31.116096 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.116657 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.117790 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.118058 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.118138 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:31.120692 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:45327: BlockingRecv error: recv got EOF from 127.6.152.131:45327 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 12:29:31 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:31 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:31.750710 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:31.744970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:44933)
0806 12:29:31.745262 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:31.745275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:31.745365 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:31.745604 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:31.745606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:31.745780 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:31.745878 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:31.745882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.746634 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.746636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:31.747360 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:31.747373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.747955 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.747958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:31.748054 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:31.748700 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:31.748712 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:31.750527 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
W20260806 12:29:31.750770 16416 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:31.745039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:44933 (local address 127.6.152.190:33591)
0806 12:29:31.745306 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:31.745310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:31.745323 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:31.745446 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:31.745449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:31.745491 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:31.745566 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:31.746023 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.746541 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.747485 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.747836 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.747949 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:31.750650 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:44933: BlockingRecv error: recv got EOF from 127.6.152.129:44933 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":68,"threads_started":1}
Aug 06 12:29:31 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:31 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:31.939684 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:31.934096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:51593)
0806 12:29:31.934368 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:31.934382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:31.934480 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:31.934709 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:31.934711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:31.934903 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:31.934999 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:31.935003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.936068 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.936071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:31.936612 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:31.936620 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.937011 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.937014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:31.937084 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:31.937566 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:31.937578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:31.939475 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":68,"threads_started":1}
W20260806 12:29:31.939719 16416 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:31.934170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:51593 (local address 127.6.152.190:33591)
0806 12:29:31.934358 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:31.934362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:31.934450 (+ 88us) server_negotiation.cc:407] Connection header received
0806 12:29:31.934558 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:31.934560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:31.934603 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:31.934669 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:31.935146 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.935948 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.936709 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:31.936917 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:31.936991 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:31.939617 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:51593: BlockingRecv error: recv got EOF from 127.6.152.130:51593 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 12:29:32 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:32 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:32.132524 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:32.121356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:53417)
0806 12:29:32.121678 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:32.121694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:32.121831 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:32.122108 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:32.122111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:32.122331 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:32.122425 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:32.122431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.123289 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.123293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:32.123868 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:32.123876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.124409 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.124412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:32.124496 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:32.130448 (+ 5952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:32.130460 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:32.132382 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":111,"threads_started":1}
W20260806 12:29:32.132691 16416 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:32.121524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:53417 (local address 127.6.152.190:33591)
0806 12:29:32.121678 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:32.121685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:32.121771 (+ 86us) server_negotiation.cc:407] Connection header received
0806 12:29:32.121933 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:32.121936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:32.121989 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:32.122080 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:32.122565 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.123171 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.124009 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.124277 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.124391 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:32.132538 (+ 8147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:53417: BlockingRecv error: recv got EOF from 127.6.152.131:53417 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 12:29:32 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:32 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:32.756438 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:32.751460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:33279)
0806 12:29:32.751730 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:32.751744 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:32.751831 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:32.752072 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:32.752074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:32.752252 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:32.752348 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:32.752353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.753087 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.753090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:32.753627 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:32.753634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.754036 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.754039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:32.754100 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:32.754584 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:32.754596 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:32.756327 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
W20260806 12:29:32.756592 16420 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:32.751543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:33279 (local address 127.6.152.190:33591)
0806 12:29:32.751770 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:32.751774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:32.751787 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:32.751916 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:32.751918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:32.751960 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:32.752040 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:32.752447 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.752993 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.753725 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.753942 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.754017 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:32.756462 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:33279: BlockingRecv error: recv got EOF from 127.6.152.129:33279 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
Aug 06 12:29:32 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:32 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:32.946027 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:32.940487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:38555)
0806 12:29:32.940756 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:32.940769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:32.940865 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:32.941102 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:32.941104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:32.941344 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:32.941454 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:32.941460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.942199 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.942213 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:32.942829 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:32.942861 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.943488 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.943492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:32.943573 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:32.944090 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:32.944101 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:32.945915 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
W20260806 12:29:32.946303 16420 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:32.940582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:38555 (local address 127.6.152.190:33591)
0806 12:29:32.940721 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:32.940725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:32.940832 (+ 107us) server_negotiation.cc:407] Connection header received
0806 12:29:32.940950 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:32.940952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:32.940995 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:32.941063 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:32.941563 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.942104 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.943020 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:32.943348 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:32.943463 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:32.946061 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:38555: BlockingRecv error: recv got EOF from 127.6.152.130:38555 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 12:29:33 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:33 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:33.138751 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:33.133228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:43355)
0806 12:29:33.133569 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:33.133582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:33.133677 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:33.133918 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:33.133920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:33.134108 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:33.134205 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:33.134210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.134946 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.134949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:33.135542 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:33.135549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.135965 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.135967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:33.136027 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:33.136511 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:33.136523 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:33.138560 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":106,"threads_started":1}
W20260806 12:29:33.138787 16420 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:33.133346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:43355 (local address 127.6.152.190:33591)
0806 12:29:33.133482 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:33.133485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:33.133632 (+ 147us) server_negotiation.cc:407] Connection header received
0806 12:29:33.133759 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:33.133761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:33.133809 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:33.133868 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:33.134310 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.134850 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.135647 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.135861 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.135943 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:33.138679 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:43355: BlockingRecv error: recv got EOF from 127.6.152.131:43355 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 12:29:33 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:33 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:33.762989 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:33.757384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:41963)
0806 12:29:33.757676 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:33.757690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:33.757788 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:33.758048 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:33.758050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:33.758232 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:33.758332 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:33.758337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.759223 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.759227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:33.759816 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:33.759824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.760253 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.760256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:33.760321 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:33.760889 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:33.760910 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:33.762869 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
W20260806 12:29:33.763197 16424 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:33.757447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:41963 (local address 127.6.152.190:33591)
0806 12:29:33.757703 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:33.757707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:33.757739 (+ 32us) server_negotiation.cc:407] Connection header received
0806 12:29:33.757875 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:33.757877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:33.757926 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:33.758008 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:33.758446 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.759075 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.759917 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.760140 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.760221 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:33.763017 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:41963: BlockingRecv error: recv got EOF from 127.6.152.129:41963 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":63,"threads_started":1}
Aug 06 12:29:33 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:33 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:33.952903 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:33.946836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:36867)
0806 12:29:33.947201 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:33.947216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:33.947352 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:33.947639 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:33.947642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:33.947853 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:33.947949 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:33.947955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.948846 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.948851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:33.949424 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:33.949432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.949923 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.949927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:33.950015 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:33.950576 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:33.950592 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:33.952790 (+ 2198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
W20260806 12:29:33.953044 16424 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:33.946969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:36867 (local address 127.6.152.190:33591)
0806 12:29:33.947153 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:33.947158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:33.947297 (+ 139us) server_negotiation.cc:407] Connection header received
0806 12:29:33.947458 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:33.947463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:33.947520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:33.947610 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:33.948086 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.948727 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.949560 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:33.949797 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:33.949918 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:33.952929 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:36867: BlockingRecv error: recv got EOF from 127.6.152.130:36867 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 12:29:34 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:34 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:34.145157 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:34.139713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:38611)
0806 12:29:34.139990 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:34.140004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:34.140114 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:34.140369 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:34.140371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:34.140547 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:34.140641 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:34.140647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.141441 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.141444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:34.142143 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:34.142152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.142576 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.142578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:34.142639 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:34.143214 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:34.143226 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:34.145042 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
W20260806 12:29:34.145470 16424 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:34.139778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:38611 (local address 127.6.152.190:33591)
0806 12:29:34.139890 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:34.139894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:34.140088 (+ 194us) server_negotiation.cc:407] Connection header received
0806 12:29:34.140180 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:34.140182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:34.140229 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:34.140297 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:34.140778 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.141331 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.142252 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.142473 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.142562 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:34.145233 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:38611: BlockingRecv error: recv got EOF from 127.6.152.131:38611 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 12:29:34 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:34 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:34.769855 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:34.764118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:52633)
0806 12:29:34.764466 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:34.764480 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:34.764605 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:34.765159 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:34.765161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:34.765345 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:34.765442 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:34.765447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.766198 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.766200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:34.766771 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:34.766779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.767243 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.767245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:34.767307 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:34.767772 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:34.767784 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:34.769694 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
W20260806 12:29:34.769986 16428 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:34.764192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:52633 (local address 127.6.152.190:33591)
0806 12:29:34.764524 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:34.764531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:34.764548 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:29:34.764833 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:34.764837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:34.764892 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:34.765134 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:34.765546 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.766091 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.766873 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.767098 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.767195 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:34.769856 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:52633: BlockingRecv error: recv got EOF from 127.6.152.129:52633 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":134,"threads_started":1}
Aug 06 12:29:34 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:34 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:34.959359 16429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:34.953761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:34303)
0806 12:29:34.954038 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:34.954051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:34.954133 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:34.954353 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:34.954355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:34.954538 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:34.954630 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:34.954635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.955462 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.955465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:34.956019 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:34.956027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.956461 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.956463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:34.956521 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:34.957034 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:34.957045 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:34.959091 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":98,"threads_started":1}
W20260806 12:29:34.959432 16428 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:34.953969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:34303 (local address 127.6.152.190:33591)
0806 12:29:34.954116 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:34.954120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:34.954133 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:34.954201 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:34.954204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:34.954244 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:34.954323 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:34.954741 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.955359 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.956128 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:34.956364 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:34.956443 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:34.959324 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:34303: BlockingRecv error: recv got EOF from 127.6.152.130:34303 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 12:29:35 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:35 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:35.151486 16430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:35.146000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:34653)
0806 12:29:35.146418 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:35.146434 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:35.146548 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:35.146787 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:35.146790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:35.146990 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:35.147090 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:35.147096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.147903 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.147906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:35.148477 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:35.148498 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.148913 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.148915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:35.148977 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:35.149551 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:35.149564 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:35.151375 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":127,"threads_started":1}
W20260806 12:29:35.151667 16428 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:35.146033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:34653 (local address 127.6.152.190:33591)
0806 12:29:35.146201 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:35.146205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:35.146518 (+ 313us) server_negotiation.cc:407] Connection header received
0806 12:29:35.146630 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:35.146633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:35.146679 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:35.146747 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:35.147247 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.147769 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.148612 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.148815 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.148890 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:35.151509 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:34653: BlockingRecv error: recv got EOF from 127.6.152.131:34653 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 12:29:35 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:35 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:35.776494 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:35.770742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:49749)
0806 12:29:35.771022 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:35.771036 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:35.771170 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:35.771439 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:35.771441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:35.771625 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:35.771722 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:35.771727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.772455 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.772457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:35.773011 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:35.773018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.773422 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.773425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:35.773484 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:35.773973 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:35.773988 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:35.776318 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
W20260806 12:29:35.776603 16432 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:35.770825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49749 (local address 127.6.152.190:33591)
0806 12:29:35.771104 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:35.771111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:35.771154 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:29:35.771273 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:35.771275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:35.771321 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:35.771403 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:35.771826 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.772361 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.773111 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.773327 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.773402 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:35.776451 (+ 3049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:49749: BlockingRecv error: recv got EOF from 127.6.152.129:49749 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
Aug 06 12:29:35 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:35 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:35.965421 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:35.960143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:53953)
0806 12:29:35.960395 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:35.960410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:35.960521 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:35.960778 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:35.960780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:35.960979 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:35.961087 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:35.961093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.961798 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.961800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:35.962349 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:35.962357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.962758 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.962760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:35.962824 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:35.963373 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:35.963385 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:35.965251 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
W20260806 12:29:35.965482 16432 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:35.960227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:53953 (local address 127.6.152.190:33591)
0806 12:29:35.960364 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:35.960367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:35.960487 (+ 120us) server_negotiation.cc:407] Connection header received
0806 12:29:35.960597 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:35.960599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:35.960658 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:35.960742 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:35.961188 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.961704 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.962447 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:35.962661 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:35.962737 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:35.965372 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:53953: BlockingRecv error: recv got EOF from 127.6.152.130:53953 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 12:29:36 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:36 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:36.157922 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:36.152373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:48361)
0806 12:29:36.152750 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:36.152763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:36.152894 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:36.153185 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:36.153187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:36.153406 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:36.153498 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:36.153503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.154279 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.154284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:36.154859 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:36.154867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.155417 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.155421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:36.155509 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:36.155976 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:36.155986 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:36.157808 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":103,"threads_started":1}
W20260806 12:29:36.158100 16432 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:36.152451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:48361 (local address 127.6.152.190:33591)
0806 12:29:36.152603 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:36.152608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:36.152850 (+ 242us) server_negotiation.cc:407] Connection header received
0806 12:29:36.153008 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:36.153012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:36.153069 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:36.153157 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:36.153623 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.154164 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.154991 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.155298 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.155417 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:36.157942 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:48361: BlockingRecv error: recv got EOF from 127.6.152.131:48361 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 12:29:36 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:36 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:36.782826 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:36.777206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:45205)
0806 12:29:36.777558 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:36.777572 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:36.777662 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:36.777937 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:36.777939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:36.778144 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:36.778245 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:36.778250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.779000 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.779003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:36.779586 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:36.779593 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.780024 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.780026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:36.780089 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:36.780604 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:36.780618 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:36.782655 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":69,"threads_started":1}
W20260806 12:29:36.782919 16436 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:36.777305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:45205 (local address 127.6.152.190:33591)
0806 12:29:36.777610 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:36.777613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:36.777627 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:36.777750 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:36.777752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:36.777796 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:36.777871 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:36.778346 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.778906 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.779696 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.779922 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.780008 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:36.782788 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:45205: BlockingRecv error: recv got EOF from 127.6.152.129:45205 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":122,"threads_started":1}
Aug 06 12:29:36 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:36 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:36.971751 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:36.966232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:37675)
0806 12:29:36.966566 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:36.966580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:36.966678 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:36.966914 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:36.966916 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:36.967099 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:36.967226 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:36.967231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.967956 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.967958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:36.968504 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:36.968510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.968922 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.968925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:36.968989 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:36.969542 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:36.969556 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:36.971637 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":122,"threads_started":1}
W20260806 12:29:36.971938 16436 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:36.966285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:37675 (local address 127.6.152.190:33591)
0806 12:29:36.966443 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:36.966447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:36.966656 (+ 209us) server_negotiation.cc:407] Connection header received
0806 12:29:36.966756 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:36.966759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:36.966802 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:36.966870 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:36.967319 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.967845 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.968602 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:36.968815 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:36.968890 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:36.971780 (+ 2890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:37675: BlockingRecv error: recv got EOF from 127.6.152.130:37675 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 12:29:37 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:37 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:37.169402 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:37.158776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:35019)
0806 12:29:37.159099 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:37.159156 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:37.159270 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:37.159579 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:37.159583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:37.159837 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:37.159991 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:37.160003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.160753 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.160756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:37.161321 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:37.161329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.161749 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.161752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:37.161815 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:37.167282 (+ 5467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:37.167294 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:37.169187 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":72,"threads_started":1}
W20260806 12:29:37.169425 16436 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:37.158820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:35019 (local address 127.6.152.190:33591)
0806 12:29:37.159089 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:37.159096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:37.159219 (+ 123us) server_negotiation.cc:407] Connection header received
0806 12:29:37.159370 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:37.159373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:37.159435 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:37.159516 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:37.160123 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.160650 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.161426 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.161650 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.161728 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:37.169321 (+ 7593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:35019: BlockingRecv error: recv got EOF from 127.6.152.131:35019 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 12:29:37 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:37 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:37.789492 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:37.783946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:59867)
0806 12:29:37.784240 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:37.784254 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:37.784340 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:37.784584 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:37.784586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:37.784766 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:37.784864 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:37.784869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.785647 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.785649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:37.786221 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:37.786229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.786640 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.786642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:37.786703 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:37.787320 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:37.787334 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:37.789373 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":112,"threads_started":1}
W20260806 12:29:37.789646 16440 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:37.784074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:59867 (local address 127.6.152.190:33591)
0806 12:29:37.784314 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:37.784317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:37.784331 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:37.784423 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:37.784426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:37.784471 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:37.784550 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:37.784982 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.785551 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.786317 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.786542 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.786629 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:37.789512 (+ 2883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:59867: BlockingRecv error: recv got EOF from 127.6.152.129:59867 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
Aug 06 12:29:37 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:37 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:37.977985 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:37.972565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:56109)
0806 12:29:37.972841 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:37.972855 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:37.972956 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:37.973175 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:37.973178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:37.973360 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:37.973460 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:37.973464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.974316 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.974319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:37.974873 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:37.974881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.975359 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.975362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:37.975426 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:37.975990 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:37.976003 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:37.977869 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":95,"threads_started":1}
W20260806 12:29:37.978132 16440 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:37.972605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:56109 (local address 127.6.152.190:33591)
0806 12:29:37.972761 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:37.972766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:37.972927 (+ 161us) server_negotiation.cc:407] Connection header received
0806 12:29:37.973022 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:37.973025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:37.973069 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:37.973137 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:37.973556 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.974219 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.974978 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:37.975257 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:37.975340 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:37.978010 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:56109: BlockingRecv error: recv got EOF from 127.6.152.130:56109 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 12:29:38 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:38 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:38.176198 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:38.170228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:58951)
0806 12:29:38.170523 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:38.170545 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:38.170676 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:38.171001 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:38.171004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:38.171240 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:38.171340 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:38.171347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.172199 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.172205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:38.172771 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:38.172779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.173259 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.173263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:38.173350 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:38.173980 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:38.173992 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:38.176041 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
W20260806 12:29:38.176306 16440 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:38.170287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58951 (local address 127.6.152.190:33591)
0806 12:29:38.170452 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:38.170457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:38.170619 (+ 162us) server_negotiation.cc:407] Connection header received
0806 12:29:38.170798 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:38.170801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:38.170890 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:38.170972 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:38.171474 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.172083 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.172891 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.173139 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.173254 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:38.176186 (+ 2932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:58951: BlockingRecv error: recv got EOF from 127.6.152.131:58951 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 12:29:38 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:38 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:38.796032 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:38.790425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:40631)
0806 12:29:38.790726 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:38.790740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:38.790836 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:38.791156 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:38.791159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:38.791340 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:38.791464 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:38.791470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.792220 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.792223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:38.792776 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:38.792783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.793205 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.793207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:38.793266 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:38.793766 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:38.793778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:38.795827 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
W20260806 12:29:38.796151 16444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:38.790482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:40631 (local address 127.6.152.190:33591)
0806 12:29:38.790757 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:38.790760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:38.790791 (+ 31us) server_negotiation.cc:407] Connection header received
0806 12:29:38.790924 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:38.790926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:38.790969 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:38.791055 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:38.791580 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.792124 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.792880 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.793109 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.793185 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:38.796017 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:40631: BlockingRecv error: recv got EOF from 127.6.152.129:40631 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
Aug 06 12:29:38 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:38 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 12:29:38.985019 16444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:38.978908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:57711 (local address 127.6.152.190:33591)
0806 12:29:38.979061 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:38.979066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:38.979415 (+ 349us) server_negotiation.cc:407] Connection header received
0806 12:29:38.979569 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:38.979572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:38.979640 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:38.979711 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:38.980262 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.980876 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.981773 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.982013 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.982122 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:38.984922 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:57711: BlockingRecv error: recv got EOF from 127.6.152.130:57711 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:29:38.985024 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:38.978851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:57711)
0806 12:29:38.979264 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:38.979280 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:38.979458 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:38.979766 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:38.979769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:38.980051 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:38.980153 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:38.980159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.981026 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.981029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:38.981628 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:38.981636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:38.982125 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:38.982129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:38.982221 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:38.982706 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:38.982717 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:38.984787 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":98,"threads_started":1}
Aug 06 12:29:39 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:39 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:39.183677 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:39.176887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:59671)
0806 12:29:39.177248 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:39.177262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:39.177410 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:39.177923 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:39.177926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:39.178161 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:39.178261 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:39.178267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.179177 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.179186 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:39.179842 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:39.179852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.180389 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.180393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:39.180482 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:39.181167 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:39.181180 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:39.183491 (+ 2311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
W20260806 12:29:39.183745 16444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:39.177003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:59671 (local address 127.6.152.190:33591)
0806 12:29:39.177167 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:39.177172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:39.177344 (+ 172us) server_negotiation.cc:407] Connection header received
0806 12:29:39.177720 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:39.177725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:39.177794 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:39.177892 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:39.178396 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.179017 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.179990 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.180259 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.180385 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:39.183625 (+ 3240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:59671: BlockingRecv error: recv got EOF from 127.6.152.131:59671 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 12:29:39 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:39 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:39.802228 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:39.796889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:32891)
0806 12:29:39.797200 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:39.797214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:39.797310 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:39.797583 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:39.797585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:39.797768 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:39.797863 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:39.797867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.798597 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.798599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:39.799180 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:39.799188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.799645 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.799647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:39.799703 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:39.800195 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:39.800206 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:39.802071 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":72,"threads_started":1}
W20260806 12:29:39.802345 16448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:39.796940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:32891 (local address 127.6.152.190:33591)
0806 12:29:39.797226 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:39.797230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:39.797266 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:29:39.797384 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:39.797386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:39.797462 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:39.797535 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:39.797967 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.798505 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.799275 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.799533 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.799614 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:39.802210 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:32891: BlockingRecv error: recv got EOF from 127.6.152.129:32891 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":103,"threads_started":1}
Aug 06 12:29:39 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:39 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:39.991192 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:39.985886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:34105)
0806 12:29:39.986210 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:39.986224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:39.986346 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:39.986619 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:39.986621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:39.986806 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:39.986904 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:39.986909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.987658 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.987660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:39.988225 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:39.988233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.988654 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.988657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:39.988721 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:39.989246 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:39.989258 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:39.991052 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
W20260806 12:29:39.991391 16448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:39.986004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:34105 (local address 127.6.152.190:33591)
0806 12:29:39.986303 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:39.986309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:39.986324 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:29:39.986439 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:39.986443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:39.986501 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:39.986573 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:39.987016 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.987557 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.988333 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:39.988554 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:39.988630 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:39.991261 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:34105: BlockingRecv error: recv got EOF from 127.6.152.130:34105 (error 108)
Metrics: {"server-negotiator.queue_time_us":207}
Aug 06 12:29:40 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:40 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:40.189776 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:40.184490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:45075)
0806 12:29:40.184757 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:40.184771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:40.184863 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:40.185109 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:40.185111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:40.185310 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:40.185424 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:40.185431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.186208 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.186210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:40.186763 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:40.186770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.187253 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.187256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:40.187321 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:40.187824 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:40.187836 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:40.189647 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
W20260806 12:29:40.189894 16448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:40.184638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:45075 (local address 127.6.152.190:33591)
0806 12:29:40.184774 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:40.184778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:40.184837 (+ 59us) server_negotiation.cc:407] Connection header received
0806 12:29:40.184943 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:40.184946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:40.184991 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:40.185068 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:40.185529 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.186113 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.186859 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.187107 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.187255 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:40.189778 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:45075: BlockingRecv error: recv got EOF from 127.6.152.131:45075 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 12:29:40 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:40 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:40.808375 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:40.803010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:58257)
0806 12:29:40.803304 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:40.803329 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:40.803465 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:40.803725 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:40.803728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:40.803920 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:40.804022 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:40.804027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.804755 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.804757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:40.805308 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:40.805315 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.805739 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.805742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:40.805803 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:40.806365 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:40.806377 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:40.808259 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":127,"threads_started":1}
W20260806 12:29:40.808527 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:40.803057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:58257 (local address 127.6.152.190:33591)
0806 12:29:40.803347 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:40.803353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:40.803436 (+ 83us) server_negotiation.cc:407] Connection header received
0806 12:29:40.803561 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:40.803564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:40.803609 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:40.803683 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:40.804135 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.804663 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.805414 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.805637 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.805715 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:40.808392 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:58257: BlockingRecv error: recv got EOF from 127.6.152.129:58257 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
Aug 06 12:29:40 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:40 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:40.997475 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:40.992099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:39895)
0806 12:29:40.992380 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:40.992394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:40.992495 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:40.992730 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:40.992732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:40.992911 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:40.993007 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:40.993012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.993728 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.993730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:40.994280 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:40.994287 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.994704 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.994707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:40.994769 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:40.995376 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:40.995388 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:40.997318 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":88,"threads_started":1}
W20260806 12:29:40.997710 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:40.992198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:39895 (local address 127.6.152.190:33591)
0806 12:29:40.992394 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:40.992397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:40.992468 (+ 71us) server_negotiation.cc:407] Connection header received
0806 12:29:40.992577 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:40.992579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:40.992624 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:40.992692 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:40.993115 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.993636 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.994397 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:40.994608 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:40.994685 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:40.997487 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:39895: BlockingRecv error: recv got EOF from 127.6.152.130:39895 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 12:29:41 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:41 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:41.196455 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:41.190783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:45833)
0806 12:29:41.191180 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:41.191201 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:41.191320 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:41.191641 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:41.191644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:41.191881 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:41.191995 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:41.192002 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.192766 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.192768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:41.193317 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:41.193324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.193759 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.193761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:41.193820 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:41.194397 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:41.194410 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:41.196341 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":89,"threads_started":1}
W20260806 12:29:41.196666 16452 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:41.190844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:45833 (local address 127.6.152.190:33591)
0806 12:29:41.190993 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:41.190998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:41.191284 (+ 286us) server_negotiation.cc:407] Connection header received
0806 12:29:41.191442 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:41.191448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:41.191506 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:41.191590 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:41.192146 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.192672 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.193435 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.193657 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.193737 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:41.196513 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:45833: BlockingRecv error: recv got EOF from 127.6.152.131:45833 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 12:29:41 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:41 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:41.814637 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:41.809251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:47237)
0806 12:29:41.809567 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:41.809582 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:41.809676 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:41.809936 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:41.809938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:41.810126 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:41.810237 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:41.810242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.810980 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.810982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:41.811618 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:41.811628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.812039 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.812042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:41.812102 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:41.812606 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:41.812617 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:41.814512 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":69,"threads_started":1}
W20260806 12:29:41.814792 16457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:41.809344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:47237 (local address 127.6.152.190:33591)
0806 12:29:41.809652 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:41.809656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:41.809669 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:41.809767 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:41.809770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:41.809815 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:41.809893 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:41.810339 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.810885 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.811724 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:41.811938 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:41.812015 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:41.814651 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:47237: BlockingRecv error: recv got EOF from 127.6.152.129:47237 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":83,"threads_started":1}
Aug 06 12:29:41 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:42 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:42.004029 16458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:41.998256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:36955)
0806 12:29:41.998567 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:41.998580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:41.998697 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:41.998939 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:41.998941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:41.999158 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:41.999255 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:41.999261 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.000327 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.000330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:42.000957 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:42.000967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.001390 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.001392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:42.001485 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:42.001999 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:42.002011 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:42.003912 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
W20260806 12:29:42.004227 16457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:41.998355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:36955 (local address 127.6.152.190:33591)
0806 12:29:41.998518 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:41.998522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:41.998650 (+ 128us) server_negotiation.cc:407] Connection header received
0806 12:29:41.998778 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:41.998780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:41.998826 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:41.998905 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:41.999374 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.000191 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.001074 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.001292 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.001373 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:42.004085 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:36955: BlockingRecv error: recv got EOF from 127.6.152.130:36955 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 12:29:42 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:42 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:42.207515 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:42.197242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:46303)
0806 12:29:42.197559 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:42.197572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:42.197735 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:42.198047 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:42.198050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:42.198287 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:42.198382 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:42.198388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.199200 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.199205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:42.199744 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:42.199752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.200241 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.200245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:42.200332 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:42.205200 (+ 4868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:42.205213 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:42.207334 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
W20260806 12:29:42.207584 16457 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:42.197325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:46303 (local address 127.6.152.190:33591)
0806 12:29:42.197498 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:42.197503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:42.197662 (+ 159us) server_negotiation.cc:407] Connection header received
0806 12:29:42.197836 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:42.197840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:42.197893 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:42.198021 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:42.198516 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.199057 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.199869 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.200118 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.200225 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:42.207467 (+ 7242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:46303: BlockingRecv error: recv got EOF from 127.6.152.131:46303 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 12:29:42 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:42 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:42.821455 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:42.815633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:59673)
0806 12:29:42.815927 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:42.815941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:42.816037 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:42.816291 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:42.816293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:42.816477 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:42.816583 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:42.816588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.817323 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.817326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:42.817918 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:42.817926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.818353 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.818356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:42.818418 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:42.818999 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:42.819012 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:42.821260 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
W20260806 12:29:42.821514 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:42.815689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:59673 (local address 127.6.152.190:33591)
0806 12:29:42.815995 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:42.815999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:42.816013 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:29:42.816128 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:42.816131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:42.816177 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:42.816256 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:42.816688 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.817227 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.818021 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:42.818239 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:42.818314 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:42.821392 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:59673: BlockingRecv error: recv got EOF from 127.6.152.129:59673 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":98,"threads_started":1}
Aug 06 12:29:43 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:43 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:43.010669 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:43.004974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:45807)
0806 12:29:43.005456 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:43.005473 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:43.005596 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:43.005882 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:43.005885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:43.006164 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:43.006285 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:43.006292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.007045 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.007047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:43.007630 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:43.007638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.008080 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.008083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:43.008144 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:43.008633 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:43.008644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:43.010557 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":89,"threads_started":1}
W20260806 12:29:43.010823 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:43.005114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:45807 (local address 127.6.152.190:33591)
0806 12:29:43.005280 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:43.005287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:43.005542 (+ 255us) server_negotiation.cc:407] Connection header received
0806 12:29:43.005690 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:43.005695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:43.005759 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:43.005835 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:43.006426 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.006948 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.007747 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.007978 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.008063 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:43.010691 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:45807: BlockingRecv error: recv got EOF from 127.6.152.130:45807 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 12:29:43 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:43 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:43.214170 16463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:43.208469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:52977)
0806 12:29:43.208745 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:43.208759 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:43.208863 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:43.209103 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:43.209105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:43.209290 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:43.209385 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:43.209390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.210232 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.210236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:43.210981 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:43.210993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.211595 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.211598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:43.211687 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:43.212258 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:43.212271 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:43.214055 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":73,"threads_started":1}
W20260806 12:29:43.214350 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:43.208605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:52977 (local address 127.6.152.190:33591)
0806 12:29:43.208775 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:43.208779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:43.208834 (+ 55us) server_negotiation.cc:407] Connection header received
0806 12:29:43.208945 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:43.208947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:43.208992 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:43.209067 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:43.209507 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.210060 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.211108 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.211467 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.211586 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:43.214192 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:52977: BlockingRecv error: recv got EOF from 127.6.152.131:52977 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 12:29:43 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:43 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:43.827541 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:43.822354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:40757)
0806 12:29:43.822644 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:43.822662 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:43.822778 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:43.823011 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:43.823013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:43.823218 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:43.823333 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:43.823338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.824087 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.824089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:43.824641 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:43.824648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.825053 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.825055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:43.825127 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:43.825628 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:43.825640 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:43.827403 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":73,"threads_started":1}
W20260806 12:29:43.827684 16465 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:43.822471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:40757 (local address 127.6.152.190:33591)
0806 12:29:43.822798 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:43.822802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:43.822815 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:43.822850 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:43.822852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:43.822893 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:43.822970 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:43.823437 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.823993 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.824740 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:43.824957 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:43.825031 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:43.827552 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:40757: BlockingRecv error: recv got EOF from 127.6.152.129:40757 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":80,"threads_started":1}
Aug 06 12:29:44 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:44 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:44.017520 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:44.011706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:49667)
0806 12:29:44.011995 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:44.012008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:44.012171 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:44.012483 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:44.012487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:44.012759 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:44.012912 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:44.012921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.013881 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.013884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:44.014417 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:44.014425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.014854 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.014856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:44.014933 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:44.015433 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:44.015445 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:44.017397 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
W20260806 12:29:44.017673 16465 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:44.011781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:49667 (local address 127.6.152.190:33591)
0806 12:29:44.011953 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:44.011958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:44.012101 (+ 143us) server_negotiation.cc:407] Connection header received
0806 12:29:44.012284 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:44.012289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:44.012345 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:44.012443 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:44.013044 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.013742 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.014519 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.014755 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.014832 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:44.017542 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:49667: BlockingRecv error: recv got EOF from 127.6.152.130:49667 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 12:29:44 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:44 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.131@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:44.220839 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:44.214892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.131:34397)
0806 12:29:44.215264 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:44.215277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:44.215414 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:44.215710 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:44.215713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:44.215941 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:44.216044 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:44.216050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.216801 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.216806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:44.217358 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:44.217368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.217840 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.217844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:44.217929 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:44.218528 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:44.218542 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:44.220700 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":92,"threads_started":1}
W20260806 12:29:44.220921 16465 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:44.215012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:34397 (local address 127.6.152.190:33591)
0806 12:29:44.215215 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:44.215220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:44.215353 (+ 133us) server_negotiation.cc:407] Connection header received
0806 12:29:44.215545 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:44.215548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:44.215595 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:44.215680 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:44.216167 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.216688 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.217491 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.217724 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.217833 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:44.220829 (+ 2996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.131:34397: BlockingRecv error: recv got EOF from 127.6.152.131:34397 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 12:29:44 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:44 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.129@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:44.834093 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:44.828702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.129:49989)
0806 12:29:44.829013 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:44.829043 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:44.829174 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:44.829443 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:44.829445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:44.829642 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:44.829738 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:44.829742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.830517 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.830520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:44.831103 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:44.831110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.831571 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.831573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:44.831652 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:44.832146 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:44.832157 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:44.833954 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
W20260806 12:29:44.834231 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:44.828745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49989 (local address 127.6.152.190:33591)
0806 12:29:44.829013 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:44.829020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:44.829123 (+ 103us) server_negotiation.cc:407] Connection header received
0806 12:29:44.829276 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:44.829279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:44.829323 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:44.829399 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:44.829834 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.830393 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.831261 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:44.831476 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:44.831552 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:44.834097 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.129:49989: BlockingRecv error: recv got EOF from 127.6.152.129:49989 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":64,"threads_started":1}
Aug 06 12:29:45 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 12:29:45 dist-test-slave-1xrh krb5kdc[15364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.152.130@KRBTEST.COM for oryx/127.6.152.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 12:29:45.023970 16470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.018449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33591 (local address 127.6.152.130:45797)
0806 12:29:45.018715 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:45.018729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:45.018817 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:45.019086 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:45.019088 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:45.019290 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:45.019384 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.019389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.020102 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.020104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.020658 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.020665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.021102 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.021105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.021169 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.021734 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:45.021747 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:45.023856 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.152.190:33591: Server oryx/127.6.152.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
W20260806 12:29:45.024138 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:45.018579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:45797 (local address 127.6.152.190:33591)
0806 12:29:45.018751 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:45.018755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:45.018772 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:29:45.018921 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:45.018924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:45.018968 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:45.019034 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:45.019483 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.020009 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.020754 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.020974 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.021072 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.024013 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.152.130:45797: BlockingRecv error: recv got EOF from 127.6.152.130:45797 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 12:29:45.102197 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15973
I20260806 12:29:45.109854 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16129
I20260806 12:29:45.115823 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16270
I20260806 12:29:45.121804 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 15901
2026-08-06T12:29:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18057 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T12:29:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:45Z Disabled control of system clock
I20260806 12:29:45.153246 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:36027
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:36027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 12:29:45.263675 16475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:45.264114 16475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:45.264184 16475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:45.268922 16475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:45.268977 16475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:45.268993 16475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:45.269011 16475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:45.269062 16475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:45.273789 16475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:36027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:36027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:45.274780 16475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:45.276165 16475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:45.284557 16480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.284569 16483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.284569 16481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:45.284807 16475 server_base.cc:1034] running on GCE node
I20260806 12:29:45.285383 16475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:45.286588 16475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:45.287786 16475 hybrid_clock.cc:648] HybridClock initialized: now 1786019385287771 us; error 27 us; skew 500 ppm
I20260806 12:29:45.289570 16475 webserver.cc:460] Webserver started at http://127.6.152.190:38575/ using document root <none> and password file <none>
I20260806 12:29:45.290200 16475 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:45.290307 16475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:45.290621 16475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:45.292763 16475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "a296dc6b5a374d538a32163a6b0ecf06"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c41ff17b544b92ddfbf7442ac579bb43"
server_key_iv: "90b8da86dd0ca4204b825e80d18765b9"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.293358 16475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "a296dc6b5a374d538a32163a6b0ecf06"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c41ff17b544b92ddfbf7442ac579bb43"
server_key_iv: "90b8da86dd0ca4204b825e80d18765b9"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.297111 16475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:45.299489 16490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.300661 16475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:45.300829 16475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "a296dc6b5a374d538a32163a6b0ecf06"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c41ff17b544b92ddfbf7442ac579bb43"
server_key_iv: "90b8da86dd0ca4204b825e80d18765b9"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.301038 16475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:45.327280 16475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:45.327997 16475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:45.328202 16475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:45.335337 16475 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:36027
I20260806 12:29:45.335359 16543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:36027 every 8 connection(s)
I20260806 12:29:45.336265 16475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:45.338894 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16475
I20260806 12:29:45.339015 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:45.339076 16544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:45.339370 6754 external_mini_cluster.cc:1428] Setting key ee35db517e61b8f7d1dd6e00ef539169
I20260806 12:29:45.345325 16544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06: Bootstrap starting.
I20260806 12:29:45.347617 16544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:45.348467 16544 log.cc:824] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:45.348816 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.340854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40108 (local address 127.6.152.190:36027)
0806 12:29:45.341464 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:45.341478 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:45.341513 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:29:45.342200 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:45.342216 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:45.342593 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:45.342952 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:45.343425 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.344474 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.345400 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.345923 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.346096 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.346992 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:45.346997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:45.347008 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:45.347439 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:45.347490 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:45.347511 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:45.347648 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:45.347843 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 12:29:45.348085 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":131,"threads_started":1}
I20260806 12:29:45.350458 16544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06: No bootstrap required, opened a new log
I20260806 12:29:45.353757 16544 raft_consensus.cc:348] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a296dc6b5a374d538a32163a6b0ecf06" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36027 } }
I20260806 12:29:45.353960 16544 raft_consensus.cc:374] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:45.353993 16544 raft_consensus.cc:729] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a296dc6b5a374d538a32163a6b0ecf06, State: Initialized, Role: FOLLOWER
I20260806 12:29:45.354362 16544 consensus_queue.cc:260] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a296dc6b5a374d538a32163a6b0ecf06" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36027 } }
I20260806 12:29:45.354481 16544 raft_consensus.cc:388] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:45.354524 16544 raft_consensus.cc:482] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:45.354590 16544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:45.355509 16544 raft_consensus.cc:504] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a296dc6b5a374d538a32163a6b0ecf06" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36027 } }
I20260806 12:29:45.355888 16544 leader_election.cc:302] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a296dc6b5a374d538a32163a6b0ecf06; no voters:
I20260806 12:29:45.356191 16544 leader_election.cc:288] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:45.356309 16550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:45.356597 16550 raft_consensus.cc:686] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [term 1 LEADER]: Becoming Leader. State: Replica: a296dc6b5a374d538a32163a6b0ecf06, State: Running, Role: LEADER
I20260806 12:29:45.356891 16550 consensus_queue.cc:237] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a296dc6b5a374d538a32163a6b0ecf06" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36027 } }
I20260806 12:29:45.357332 16544 sys_catalog.cc:564] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:45.358572 16553 sys_catalog.cc:455] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a296dc6b5a374d538a32163a6b0ecf06. Latest consensus state: current_term: 1 leader_uuid: "a296dc6b5a374d538a32163a6b0ecf06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a296dc6b5a374d538a32163a6b0ecf06" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36027 } } }
I20260806 12:29:45.358707 16553 sys_catalog.cc:458] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:45.358974 16551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a296dc6b5a374d538a32163a6b0ecf06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a296dc6b5a374d538a32163a6b0ecf06" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 36027 } } }
I20260806 12:29:45.359086 16551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:45.359232 16560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:45.362151 16560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:45.367260 16560 catalog_manager.cc:1347] Generated new cluster ID: 22c33037bb324b1a8348a65202328a5e
I20260806 12:29:45.367327 16560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:45.380085 16560 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:45.380942 16560 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:45.399910 16560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a296dc6b5a374d538a32163a6b0ecf06: Generated new TSK 0
I20260806 12:29:45.400753 16560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:45.407683 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36027
--builtin_ntp_servers=127.6.152.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 12:29:45.520377 16570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:45.520641 16570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:45.520704 16570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:45.524675 16570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:45.524746 16570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:45.524834 16570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:45.529438 16570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:45.530411 16570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:45.531857 16570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:45.540992 16575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.541061 16576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.541015 16578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:45.541183 16570 server_base.cc:1034] running on GCE node
I20260806 12:29:45.541668 16570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:45.542207 16570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:45.543375 16570 hybrid_clock.cc:648] HybridClock initialized: now 1786019385543358 us; error 29 us; skew 500 ppm
I20260806 12:29:45.545151 16570 webserver.cc:460] Webserver started at http://127.6.152.129:32967/ using document root <none> and password file <none>
I20260806 12:29:45.545745 16570 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:45.545827 16570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:45.546078 16570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:45.548118 16570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "e424c90576f242b29a2a6e26f30581b9"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c5084e7073617d820248a54dbc2ff96d"
server_key_iv: "cb50771fece91edd3be527fd531f5746"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.548839 16570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "e424c90576f242b29a2a6e26f30581b9"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c5084e7073617d820248a54dbc2ff96d"
server_key_iv: "cb50771fece91edd3be527fd531f5746"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.552489 16570 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:45.554811 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.556002 16570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:45.556145 16570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "e424c90576f242b29a2a6e26f30581b9"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c5084e7073617d820248a54dbc2ff96d"
server_key_iv: "cb50771fece91edd3be527fd531f5746"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.556260 16570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:45.578452 16570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:45.579254 16570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:45.579499 16570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:45.580242 16570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:45.581231 16570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:45.581282 16570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.581328 16570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:45.581346 16570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.590698 16570 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:36525
I20260806 12:29:45.590731 16701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:36525 every 8 connection(s)
I20260806 12:29:45.591728 16570 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:45.594734 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16570
I20260806 12:29:45.594892 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:45.595237 6754 external_mini_cluster.cc:1428] Setting key ef22645a594b57a828628f679605d347
I20260806 12:29:45.597365 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36027
--builtin_ntp_servers=127.6.152.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 12:29:45.602600 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.593998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:44461 (local address 127.6.152.190:36027)
0806 12:29:45.594157 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:45.594163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:45.595345 (+ 1182us) server_negotiation.cc:407] Connection header received
0806 12:29:45.597119 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:45.597122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:45.597178 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:45.597256 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:45.598235 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.598946 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.600189 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.600609 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.600750 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.601975 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:45.601981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:45.601985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:45.602005 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:45.602033 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:45.602047 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:45.602139 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:45.602284 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 12:29:45.602367 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 12:29:45.603377 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.594436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:36027 (local address 127.6.152.129:44461)
0806 12:29:45.595188 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:45.595243 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:45.596909 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:45.597421 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:45.597428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:45.597546 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:45.598066 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.598080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.599088 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.599091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.600058 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.600068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.600800 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.600803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.600978 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.601704 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:45.601705 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:45.601744 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:45.601808 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:45.602145 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:45.602152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:45.602156 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:29:45.602249 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 12:29:45.602468 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":129,"threads_started":1}
I20260806 12:29:45.604679 16702 heartbeater.cc:347] Connected to a master server at 127.6.152.190:36027
I20260806 12:29:45.604961 16702 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:45.605655 16702 heartbeater.cc:511] Master 127.6.152.190:36027 requested a full tablet report, sending...
I20260806 12:29:45.607262 16508 ts_manager.cc:194] Registered new tserver with Master: e424c90576f242b29a2a6e26f30581b9 (127.6.152.129:36525)
I20260806 12:29:45.608613 16508 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.129:44461
W20260806 12:29:45.708611 16705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:45.708884 16705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:45.708956 16705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:45.712987 16705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:45.713074 16705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:45.713161 16705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:45.718257 16705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:45.719400 16705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:45.720809 16705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:45.730211 16711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.730216 16710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.730196 16713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:45.730551 16705 server_base.cc:1034] running on GCE node
I20260806 12:29:45.730995 16705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:45.731680 16705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:45.732880 16705 hybrid_clock.cc:648] HybridClock initialized: now 1786019385732870 us; error 32 us; skew 500 ppm
I20260806 12:29:45.734627 16705 webserver.cc:460] Webserver started at http://127.6.152.130:36815/ using document root <none> and password file <none>
I20260806 12:29:45.735256 16705 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:45.735358 16705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:45.735574 16705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:45.737534 16705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "392e166b268d4e969fc15ee953543f8f"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c7305f10920e803150ee2a8c0d251083"
server_key_iv: "39692428c6a1837f433078546f0f8c8b"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.738097 16705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "392e166b268d4e969fc15ee953543f8f"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c7305f10920e803150ee2a8c0d251083"
server_key_iv: "39692428c6a1837f433078546f0f8c8b"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.741559 16705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 12:29:45.744071 16719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.745398 16705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:45.745548 16705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "392e166b268d4e969fc15ee953543f8f"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "c7305f10920e803150ee2a8c0d251083"
server_key_iv: "39692428c6a1837f433078546f0f8c8b"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.745661 16705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:45.768824 16705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:45.769470 16705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:45.769675 16705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:45.770241 16705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:45.771193 16705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:45.771266 16705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.771332 16705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:45.771374 16705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.780592 16705 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:43553
I20260806 12:29:45.780619 16838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:43553 every 8 connection(s)
I20260806 12:29:45.781409 16705 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:45.784056 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16705
I20260806 12:29:45.784195 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:45.784500 6754 external_mini_cluster.cc:1428] Setting key ed1a753ab824aa1b7ac400a6270f3aa9
I20260806 12:29:45.786582 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36027
--builtin_ntp_servers=127.6.152.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 12:29:45.791957 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.783173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:33849 (local address 127.6.152.190:36027)
0806 12:29:45.783356 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:45.783361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:45.784797 (+ 1436us) server_negotiation.cc:407] Connection header received
0806 12:29:45.786107 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:45.786110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:45.786165 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:45.786252 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:45.787311 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.788117 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.789852 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.790090 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.790199 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.791435 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:45.791439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:45.791442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:45.791455 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:45.791483 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:45.791491 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:45.791579 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:45.791754 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 12:29:45.791819 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:29:45.792716 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.783667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:36027 (local address 127.6.152.130:33849)
0806 12:29:45.784623 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:45.784680 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:45.785920 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:45.786372 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:45.786378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:45.786455 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:45.787091 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.787106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.788281 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.788286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.789725 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.789736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.790235 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.790239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.790409 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.791162 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:45.791166 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:45.791217 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:45.791286 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:45.791582 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:45.791594 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:45.791600 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:29:45.791737 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 12:29:45.791988 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":193,"threads_started":1}
I20260806 12:29:45.793803 16839 heartbeater.cc:347] Connected to a master server at 127.6.152.190:36027
I20260806 12:29:45.794040 16839 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:45.794502 16839 heartbeater.cc:511] Master 127.6.152.190:36027 requested a full tablet report, sending...
I20260806 12:29:45.795612 16508 ts_manager.cc:194] Registered new tserver with Master: 392e166b268d4e969fc15ee953543f8f (127.6.152.130:43553)
I20260806 12:29:45.796481 16508 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.130:33849
W20260806 12:29:45.896574 16843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:45.896801 16843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:45.896835 16843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:45.900818 16843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:45.900867 16843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:45.900923 16843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:45.905723 16843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:36027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:45.906674 16843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:45.908017 16843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:29:45.917315 16843 server_base.cc:1034] running on GCE node
W20260806 12:29:45.917189 16852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.917191 16850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:45.917225 16849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:45.917852 16843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:45.918452 16843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:45.919637 16843 hybrid_clock.cc:648] HybridClock initialized: now 1786019385919619 us; error 30 us; skew 500 ppm
I20260806 12:29:45.921332 16843 webserver.cc:460] Webserver started at http://127.6.152.131:33189/ using document root <none> and password file <none>
I20260806 12:29:45.921892 16843 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:45.921988 16843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:45.922222 16843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:45.924270 16843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "05e34227213349379ab2bf0f8742506f"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "72f54add1846990dd35db85188b1013e"
server_key_iv: "7cc26abf853736cd6c20ff314f628ead"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.924832 16843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "05e34227213349379ab2bf0f8742506f"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "72f54add1846990dd35db85188b1013e"
server_key_iv: "7cc26abf853736cd6c20ff314f628ead"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.928377 16843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:45.930691 16858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.931867 16843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:45.932001 16843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "05e34227213349379ab2bf0f8742506f"
format_stamp: "Formatted at 2026-08-06 12:29:45 on dist-test-slave-1xrh"
server_key: "72f54add1846990dd35db85188b1013e"
server_key_iv: "7cc26abf853736cd6c20ff314f628ead"
server_key_version: "encryptionkey@0"
I20260806 12:29:45.932119 16843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:45.958019 16843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:45.958688 16843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:45.958895 16843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:45.959522 16843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:45.960532 16843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:45.960629 16843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.960693 16843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:45.960743 16843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:45.969393 16843 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:41951
I20260806 12:29:45.969434 16976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:41951 every 8 connection(s)
I20260806 12:29:45.970252 16843 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:45.973731 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16843
I20260806 12:29:45.973820 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:45.974109 6754 external_mini_cluster.cc:1428] Setting key 58df60f7326cb327f977927ba29b2b14
I20260806 12:29:45.979517 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.972009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:40189 (local address 127.6.152.190:36027)
0806 12:29:45.972151 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:45.972155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:45.973133 (+ 978us) server_negotiation.cc:407] Connection header received
0806 12:29:45.974670 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:45.974674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:45.974752 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:45.974860 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:45.975798 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.976494 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.977619 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.977838 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.977950 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.978992 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:45.978996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:45.978999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:45.979018 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:45.979044 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:45.979052 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:45.979162 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:45.979318 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 12:29:45.979378 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:29:45.980317 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:45.972362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:36027 (local address 127.6.152.131:40189)
0806 12:29:45.972990 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:45.973029 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:45.974447 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:45.975028 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:45.975037 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:45.975116 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:45.975658 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.975672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.976628 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.976632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.977481 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:45.977490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:45.977990 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:45.977994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:45.978173 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:45.978730 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:45.978732 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:45.978769 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:45.978827 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:45.979155 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:45.979164 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:45.979171 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 12:29:45.979303 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 12:29:45.979580 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260806 12:29:45.981364 16977 heartbeater.cc:347] Connected to a master server at 127.6.152.190:36027
I20260806 12:29:45.981608 16977 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:45.982118 16977 heartbeater.cc:511] Master 127.6.152.190:36027 requested a full tablet report, sending...
I20260806 12:29:45.983155 16508 ts_manager.cc:194] Registered new tserver with Master: 05e34227213349379ab2bf0f8742506f (127.6.152.131:41951)
I20260806 12:29:45.983736 16508 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.131:40189
I20260806 12:29:45.988004 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 12:29:45.990705 16548 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:45.989915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.6.152.190:36027)
0806 12:29:45.990028 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:45.990033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:45.990198 (+ 165us) server_negotiation.cc:407] Connection header received
0806 12:29:45.990292 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:45.990294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:45.990328 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:45.990390 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:45.990616 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40110: BlockingRecv error: recv got EOF from 127.0.0.1:40110 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:29:45.990938 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16570
I20260806 12:29:46.000459 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16705
I20260806 12:29:46.007169 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16843
I20260806 12:29:46.013373 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16475
2026-08-06T12:29:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (890 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T12:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:46Z Disabled control of system clock
I20260806 12:29:46.043783 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37929
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:36015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 12:29:46.153215 16988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:46.153487 16988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:46.153525 16988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:46.157471 16988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:46.157521 16988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:46.157537 16988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:46.157553 16988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:46.157568 16988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:46.162250 16988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:46.163281 16988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:46.164623 16988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:46.172824 16993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:46.172878 16996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:46.172816 16994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:46.173116 16988 server_base.cc:1034] running on GCE node
I20260806 12:29:46.173743 16988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:46.174845 16988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:46.175999 16988 hybrid_clock.cc:648] HybridClock initialized: now 1786019386175967 us; error 45 us; skew 500 ppm
I20260806 12:29:46.177809 16988 webserver.cc:460] Webserver started at http://127.6.152.190:35139/ using document root <none> and password file <none>
I20260806 12:29:46.178361 16988 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:46.178454 16988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:46.178695 16988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:46.180693 16988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "3faee070f9c142fea849c0620b4d987b"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "f3b96bbddb5ceb7fda89f9f9b90c0fc6"
server_key_iv: "6d353b970601dd845ba09c35f2c84f46"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.181257 16988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "3faee070f9c142fea849c0620b4d987b"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "f3b96bbddb5ceb7fda89f9f9b90c0fc6"
server_key_iv: "6d353b970601dd845ba09c35f2c84f46"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.184742 16988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 12:29:46.186970 17002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.188226 16988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:46.188367 16988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "3faee070f9c142fea849c0620b4d987b"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "f3b96bbddb5ceb7fda89f9f9b90c0fc6"
server_key_iv: "6d353b970601dd845ba09c35f2c84f46"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.188483 16988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:46.192963 16988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:46.193574 16988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:46.193754 16988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:46.200857 16988 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37929
I20260806 12:29:46.200871 17057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37929 every 8 connection(s)
I20260806 12:29:46.201735 16988 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:46.204873 17058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:46.209112 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 16988
I20260806 12:29:46.209265 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:46.209539 6754 external_mini_cluster.cc:1428] Setting key d9934197f176c155f0a3d3d3932625ec
I20260806 12:29:46.210347 17058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b: Bootstrap starting.
I20260806 12:29:46.212667 17058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:46.213549 17058 log.cc:824] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:46.215512 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.210913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.6.152.190:37929)
0806 12:29:46.211572 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:46.211587 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:46.211620 (+ 33us) server_negotiation.cc:407] Connection header received
0806 12:29:46.212224 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:46.212240 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:46.212588 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:46.212935 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:46.213857 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:46.213862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:46.213876 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:46.213911 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:46.213981 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:46.214011 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:46.214144 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:46.214384 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 12:29:46.214630 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":158,"threads_started":1}
I20260806 12:29:46.215785 17058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b: No bootstrap required, opened a new log
I20260806 12:29:46.219460 17058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3faee070f9c142fea849c0620b4d987b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37929 } }
I20260806 12:29:46.219683 17058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:46.219723 17058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3faee070f9c142fea849c0620b4d987b, State: Initialized, Role: FOLLOWER
I20260806 12:29:46.220054 17058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3faee070f9c142fea849c0620b4d987b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37929 } }
I20260806 12:29:46.220156 17058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:46.220201 17058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:46.220266 17058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:46.221053 17058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3faee070f9c142fea849c0620b4d987b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37929 } }
I20260806 12:29:46.221325 17058 leader_election.cc:302] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3faee070f9c142fea849c0620b4d987b; no voters:
I20260806 12:29:46.221585 17058 leader_election.cc:288] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:46.221706 17063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:46.221974 17063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [term 1 LEADER]: Becoming Leader. State: Replica: 3faee070f9c142fea849c0620b4d987b, State: Running, Role: LEADER
I20260806 12:29:46.222262 17063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3faee070f9c142fea849c0620b4d987b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37929 } }
I20260806 12:29:46.222831 17058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:46.224051 17064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3faee070f9c142fea849c0620b4d987b. Latest consensus state: current_term: 1 leader_uuid: "3faee070f9c142fea849c0620b4d987b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3faee070f9c142fea849c0620b4d987b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37929 } } }
I20260806 12:29:46.224191 17064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:46.224062 17065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3faee070f9c142fea849c0620b4d987b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3faee070f9c142fea849c0620b4d987b" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37929 } } }
I20260806 12:29:46.224371 17065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:46.224582 17073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:46.227252 17073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:46.232224 17073 catalog_manager.cc:1347] Generated new cluster ID: 12e7d11eff494166a36f21a1ea913348
I20260806 12:29:46.232285 17073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:46.243095 17073 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:46.243397 17073 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:46.252727 17073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3faee070f9c142fea849c0620b4d987b: Generated new TSK 0
I20260806 12:29:46.253341 17073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:29:46.258872 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37929
--builtin_ntp_servers=127.6.152.148:36015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 12:29:46.378762 17084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:46.379036 17084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:46.379105 17084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:46.383040 17084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:46.383123 17084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:46.383222 17084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:46.388060 17084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:46.389061 17084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:46.390376 17084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:46.399950 17092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:46.399914 17089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:46.399899 17090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:46.399978 17084 server_base.cc:1034] running on GCE node
I20260806 12:29:46.400511 17084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:46.401110 17084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:46.402271 17084 hybrid_clock.cc:648] HybridClock initialized: now 1786019386402249 us; error 33 us; skew 500 ppm
I20260806 12:29:46.404139 17084 webserver.cc:460] Webserver started at http://127.6.152.129:41457/ using document root <none> and password file <none>
I20260806 12:29:46.404690 17084 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:46.404755 17084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:46.405007 17084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:46.406934 17084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "0da48a72b7e54f31a0c557845fa8b780"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "c61bf94d811c291032c8855e4eb5230b"
server_key_iv: "d74284bcfc52e3707034caf92d8c7ac7"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.407559 17084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "0da48a72b7e54f31a0c557845fa8b780"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "c61bf94d811c291032c8855e4eb5230b"
server_key_iv: "d74284bcfc52e3707034caf92d8c7ac7"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.410912 17084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:29:46.413342 17099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.414768 17084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:29:46.414947 17084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "0da48a72b7e54f31a0c557845fa8b780"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "c61bf94d811c291032c8855e4eb5230b"
server_key_iv: "d74284bcfc52e3707034caf92d8c7ac7"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.415071 17084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:46.419405 17084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:46.419989 17084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:46.420176 17084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:46.420691 17084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:46.421568 17084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:46.421638 17084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.421702 17084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:46.421747 17084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.430636 17084 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:35829
I20260806 12:29:46.430663 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:35829 every 8 connection(s)
I20260806 12:29:46.431490 17084 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:46.437258 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17084
I20260806 12:29:46.437373 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:46.437361 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.433240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:32861 (local address 127.6.152.190:37929)
0806 12:29:46.433412 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:46.433417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:46.434429 (+ 1012us) server_negotiation.cc:407] Connection header received
0806 12:29:46.435768 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:46.435771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:46.435810 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:46.435887 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:46.436900 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:46.436903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:46.436905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:46.436917 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:46.436938 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:46.436946 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:46.437022 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:46.437189 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 12:29:46.437225 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 12:29:46.437686 6754 external_mini_cluster.cc:1428] Setting key ec31d367ab36033a18e2af74649f0921
I20260806 12:29:46.438308 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.433645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37929 (local address 127.6.152.129:32861)
0806 12:29:46.434283 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:46.434321 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:46.435554 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:46.436056 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:46.436062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:46.436142 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:46.436662 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:46.436664 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:46.436700 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:46.436761 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:46.437035 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:46.437047 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:46.437056 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 12:29:46.437231 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 12:29:46.437479 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260806 12:29:46.439532 17216 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37929
I20260806 12:29:46.439761 17216 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:46.439816 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37929
--builtin_ntp_servers=127.6.152.148:36015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 12:29:46.440196 17216 heartbeater.cc:511] Master 127.6.152.190:37929 requested a full tablet report, sending...
I20260806 12:29:46.441846 17019 ts_manager.cc:194] Registered new tserver with Master: 0da48a72b7e54f31a0c557845fa8b780 (127.6.152.129:35829)
W20260806 12:29:46.549230 17220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:46.549464 17220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:46.549502 17220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:46.553331 17220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:46.553371 17220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:46.553423 17220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:46.557957 17220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:46.558862 17220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:46.560145 17220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:46.569213 17226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:46.569319 17228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:46.569356 17220 server_base.cc:1034] running on GCE node
W20260806 12:29:46.569600 17225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:46.569999 17220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:46.570618 17220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:46.571776 17220 hybrid_clock.cc:648] HybridClock initialized: now 1786019386571755 us; error 31 us; skew 500 ppm
I20260806 12:29:46.573511 17220 webserver.cc:460] Webserver started at http://127.6.152.130:39489/ using document root <none> and password file <none>
I20260806 12:29:46.574060 17220 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:46.574122 17220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:46.574370 17220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:46.576368 17220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "cce3ced36cbe4f39916cfee834ffd649"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "4e8528266299866f34ed0ae544e61566"
server_key_iv: "eae40ea1402ec8850cac17805b3dc8bb"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.576933 17220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "cce3ced36cbe4f39916cfee834ffd649"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "4e8528266299866f34ed0ae544e61566"
server_key_iv: "eae40ea1402ec8850cac17805b3dc8bb"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.580477 17220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:29:46.582773 17234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.584003 17220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:46.584122 17220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "cce3ced36cbe4f39916cfee834ffd649"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "4e8528266299866f34ed0ae544e61566"
server_key_iv: "eae40ea1402ec8850cac17805b3dc8bb"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.584224 17220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:46.588726 17220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:46.589293 17220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:46.589475 17220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:46.589991 17220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:46.590862 17220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:46.590909 17220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.590991 17220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:46.591027 17220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.600034 17220 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:33399
I20260806 12:29:46.600147 17350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:33399 every 8 connection(s)
I20260806 12:29:46.600903 17220 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:46.606693 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.602673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:43249 (local address 127.6.152.190:37929)
0806 12:29:46.602809 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:46.602813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:46.603883 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 12:29:46.605109 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:46.605112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:46.605150 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:46.605224 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:46.606225 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:46.606228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:46.606230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:46.606242 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:46.606262 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:46.606270 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:46.606344 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:46.606470 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 12:29:46.606531 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:29:46.606827 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17220
I20260806 12:29:46.606978 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:29:46.607331 6754 external_mini_cluster.cc:1428] Setting key 64af020c48b3ac451ec720cf6ecc3f4c
I20260806 12:29:46.607643 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.603044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37929 (local address 127.6.152.130:43249)
0806 12:29:46.603708 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:46.603746 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:46.604927 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:46.605378 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:46.605384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:46.605471 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:46.605974 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:46.605976 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:46.606013 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:46.606076 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:46.606347 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:46.606353 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:46.606357 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:29:46.606485 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 12:29:46.606681 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":114,"threads_started":1}
I20260806 12:29:46.608759 17351 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37929
I20260806 12:29:46.608987 17351 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:46.609416 17351 heartbeater.cc:511] Master 127.6.152.190:37929 requested a full tablet report, sending...
I20260806 12:29:46.609594 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37929
--builtin_ntp_servers=127.6.152.148:36015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 12:29:46.610497 17019 ts_manager.cc:194] Registered new tserver with Master: cce3ced36cbe4f39916cfee834ffd649 (127.6.152.130:33399)
W20260806 12:29:46.717454 17354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:46.717676 17354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:46.717731 17354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:46.721871 17354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:46.721921 17354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:46.721972 17354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:46.726963 17354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:36015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:46.728042 17354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:46.729336 17354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:46.738629 17362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:46.738722 17354 server_base.cc:1034] running on GCE node
W20260806 12:29:46.738646 17359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:46.738646 17360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:46.739334 17354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:46.739984 17354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:46.741178 17354 hybrid_clock.cc:648] HybridClock initialized: now 1786019386741158 us; error 48 us; skew 500 ppm
I20260806 12:29:46.743007 17354 webserver.cc:460] Webserver started at http://127.6.152.131:36185/ using document root <none> and password file <none>
I20260806 12:29:46.743635 17354 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:46.743733 17354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:46.743968 17354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:46.745999 17354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "2ed9978bc2e548919e00ceef32d9250b"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "183cb789d02f0d7d20e205d593d5d736"
server_key_iv: "251adc11ac966b379e5e93972a9db1f1"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.746577 17354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "2ed9978bc2e548919e00ceef32d9250b"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "183cb789d02f0d7d20e205d593d5d736"
server_key_iv: "251adc11ac966b379e5e93972a9db1f1"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.750097 17354 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:29:46.752477 17369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.753705 17354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:46.753795 17354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "2ed9978bc2e548919e00ceef32d9250b"
format_stamp: "Formatted at 2026-08-06 12:29:46 on dist-test-slave-1xrh"
server_key: "183cb789d02f0d7d20e205d593d5d736"
server_key_iv: "251adc11ac966b379e5e93972a9db1f1"
server_key_version: "encryptionkey@0"
I20260806 12:29:46.753901 17354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:46.758384 17354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:46.758901 17354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:46.759052 17354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:46.759577 17354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:46.760371 17354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:46.760418 17354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.760453 17354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:46.760474 17354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:46.769091 17354 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:33847
I20260806 12:29:46.769129 17485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:33847 every 8 connection(s)
I20260806 12:29:46.769939 17354 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:46.775606 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.771676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:51203 (local address 127.6.152.190:37929)
0806 12:29:46.771837 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:46.771842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:46.772833 (+ 991us) server_negotiation.cc:407] Connection header received
0806 12:29:46.774059 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:46.774061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:46.774098 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:46.774173 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:46.775216 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:46.775218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:46.775220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:29:46.775232 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:46.775251 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:29:46.775260 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:46.775333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:46.775466 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 12:29:46.775501 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:29:46.776072 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17354
I20260806 12:29:46.776204 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:46.776476 6754 external_mini_cluster.cc:1428] Setting key 32169da3fa0527570ac82fffb9fffd1c
I20260806 12:29:46.776465 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:46.772083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37929 (local address 127.6.152.131:51203)
0806 12:29:46.772697 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:46.772735 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:46.773877 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:46.774319 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:46.774325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:46.774420 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:46.774925 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:29:46.774927 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:46.774963 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:29:46.775026 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:46.775336 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:46.775343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:46.775347 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 12:29:46.775484 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 12:29:46.775656 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260806 12:29:46.777688 17486 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37929
I20260806 12:29:46.777902 17486 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:46.778340 17486 heartbeater.cc:511] Master 127.6.152.190:37929 requested a full tablet report, sending...
I20260806 12:29:46.779336 17019 ts_manager.cc:194] Registered new tserver with Master: 2ed9978bc2e548919e00ceef32d9250b (127.6.152.131:33847)
I20260806 12:29:46.790196 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 12:29:46.793126 17061 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:29:46.792199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.6.152.190:37929)
0806 12:29:46.792359 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:46.792366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:46.792497 (+ 131us) server_negotiation.cc:407] Connection header received
0806 12:29:46.792589 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:46.792591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:46.792621 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:46.792683 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:46.792970 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56368: BlockingRecv error: recv got EOF from 127.0.0.1:56368 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 12:29:46.793339 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17084
I20260806 12:29:46.799857 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17220
I20260806 12:29:46.806164 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17354
I20260806 12:29:46.812112 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 16988
2026-08-06T12:29:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (798 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:46 dist-test-slave-1xrh krb5kdc[17496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:46 dist-test-slave-1xrh krb5kdc[17496](info): set up 2 sockets
Aug 06 12:29:46 dist-test-slave-1xrh krb5kdc[17496](info): commencing operation
krb5kdc: starting...
W20260806 12:29:47.900297 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:47 dist-test-slave-1xrh krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:29:48.022756 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:40293
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:40293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:48.130125 17509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:48.130398 17509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:48.130465 17509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:48.134795 17509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:29:48.134868 17509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:48.134922 17509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:48.134966 17509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:29:48.135003 17509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:29:48.140174 17509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:40293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:40293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:48.141139 17509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:48.142062 17509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:48.150394 17517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:48.150400 17514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:48.150503 17509 server_base.cc:1034] running on GCE node
W20260806 12:29:48.150415 17515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:48.151345 17509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:48.152321 17509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:48.153512 17509 hybrid_clock.cc:648] HybridClock initialized: now 1786019388153494 us; error 28 us; skew 500 ppm
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:48.165211 17509 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:29:48.166046 17509 webserver.cc:460] Webserver started at http://127.6.152.190:35421/ using document root <none> and password file <none>
I20260806 12:29:48.166597 17509 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:48.166694 17509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:48.166927 17509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:48.168972 17509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "2be40d7e58e542bf99c8d3d949b8f786"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "a559ea1c48388926e6db37c23d7da154"
server_key_iv: "6ee093923ff887672e1ed976a32150d3"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.169560 17509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "2be40d7e58e542bf99c8d3d949b8f786"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "a559ea1c48388926e6db37c23d7da154"
server_key_iv: "6ee093923ff887672e1ed976a32150d3"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.173287 17509 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:48.175575 17524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.176723 17509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:48.176869 17509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "2be40d7e58e542bf99c8d3d949b8f786"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "a559ea1c48388926e6db37c23d7da154"
server_key_iv: "6ee093923ff887672e1ed976a32150d3"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.176975 17509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:48.205610 17509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:48.208889 17509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:48.209113 17509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:48.216315 17509 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:40293
I20260806 12:29:48.216579 17579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:40293 every 8 connection(s)
I20260806 12:29:48.217300 17509 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:29:48.218817 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17509
I20260806 12:29:48.218954 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:29:48.219326 6754 external_mini_cluster.cc:1428] Setting key 8f73c0366212a30cccf11de817578b7e
I20260806 12:29:48.220497 17580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:48.226794 17580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786: Bootstrap starting.
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:48.229444 17580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:48.230360 17580 log.cc:824] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:48.232631 17580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786: No bootstrap required, opened a new log
I20260806 12:29:48.233863 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.220702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.6.152.190:40293)
0806 12:29:48.221100 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.221116 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.221151 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:29:48.221769 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.221784 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.222090 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.222372 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:48.223289 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.224307 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.225474 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.225911 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.226027 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.228762 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:48.228768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:48.228783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:48.228814 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:48.230838 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.231370 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.231376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.231381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.231509 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.231778 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.231781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.231783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.232276 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:48.232465 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.232672 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.232981 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":129,"threads_started":1}
I20260806 12:29:48.236413 17580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be40d7e58e542bf99c8d3d949b8f786" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40293 } }
I20260806 12:29:48.236624 17580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:48.236697 17580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be40d7e58e542bf99c8d3d949b8f786, State: Initialized, Role: FOLLOWER
I20260806 12:29:48.237109 17580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be40d7e58e542bf99c8d3d949b8f786" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40293 } }
I20260806 12:29:48.237246 17580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:29:48.237306 17580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:29:48.237424 17580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:48.238221 17580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be40d7e58e542bf99c8d3d949b8f786" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40293 } }
I20260806 12:29:48.238565 17580 leader_election.cc:302] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2be40d7e58e542bf99c8d3d949b8f786; no voters:
I20260806 12:29:48.238853 17580 leader_election.cc:288] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:29:48.238984 17585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:48.239261 17585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [term 1 LEADER]: Becoming Leader. State: Replica: 2be40d7e58e542bf99c8d3d949b8f786, State: Running, Role: LEADER
I20260806 12:29:48.239562 17585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be40d7e58e542bf99c8d3d949b8f786" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40293 } }
I20260806 12:29:48.240021 17580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:29:48.241220 17587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be40d7e58e542bf99c8d3d949b8f786. Latest consensus state: current_term: 1 leader_uuid: "2be40d7e58e542bf99c8d3d949b8f786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be40d7e58e542bf99c8d3d949b8f786" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40293 } } }
I20260806 12:29:48.241351 17587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:48.241236 17586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2be40d7e58e542bf99c8d3d949b8f786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be40d7e58e542bf99c8d3d949b8f786" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 40293 } } }
I20260806 12:29:48.241823 17586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786 [sys.catalog]: This master's current role is: LEADER
I20260806 12:29:48.242687 17594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:29:48.245302 17594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:29:48.250541 17594 catalog_manager.cc:1347] Generated new cluster ID: f5a9176a9ddd417ebc9cefca17cfc3c6
I20260806 12:29:48.250607 17594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:29:48.255745 17594 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:29:48.256567 17594 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:29:48.263202 17594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786: Generated new TSK 0
I20260806 12:29:48.263885 17594 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:29:48.302098 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40293
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:48.411461 17606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:48.411712 17606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:48.411783 17606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:48.415802 17606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:48.415874 17606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:48.415957 17606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:29:48.420992 17606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:48.421947 17606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:48.422930 17606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:48.432188 17615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:48.432205 17611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:48.432189 17612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:48.432430 17606 server_base.cc:1034] running on GCE node
I20260806 12:29:48.432811 17606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:48.433383 17606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:48.434545 17606 hybrid_clock.cc:648] HybridClock initialized: now 1786019388434512 us; error 45 us; skew 500 ppm
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:48.437215 17606 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:29:48.438056 17606 webserver.cc:460] Webserver started at http://127.6.152.129:45931/ using document root <none> and password file <none>
I20260806 12:29:48.438575 17606 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:48.438676 17606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:48.438915 17606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:48.440928 17606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "5964f2b36f2c41d1a1fdf54745995c05"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "b11a02186ed5559903836af2a498bf4b"
server_key_iv: "b73a3d08dd43a4d7b0b4930b0d3d0cec"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.441577 17606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "5964f2b36f2c41d1a1fdf54745995c05"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "b11a02186ed5559903836af2a498bf4b"
server_key_iv: "b73a3d08dd43a4d7b0b4930b0d3d0cec"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.445173 17606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:29:48.447414 17624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.448557 17606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:29:48.448724 17606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "5964f2b36f2c41d1a1fdf54745995c05"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "b11a02186ed5559903836af2a498bf4b"
server_key_iv: "b73a3d08dd43a4d7b0b4930b0d3d0cec"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.448829 17606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:48.460969 17606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:48.463598 17606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:48.463781 17606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:48.464340 17606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:48.465111 17606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:48.465181 17606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.465253 17606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:48.465298 17606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.473683 17606 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:44457
I20260806 12:29:48.473714 17741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:44457 every 8 connection(s)
I20260806 12:29:48.474491 17606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:29:48.478381 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17606
I20260806 12:29:48.478507 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:29:48.478845 6754 external_mini_cluster.cc:1428] Setting key 9b30283244ff7fb329a940d88eb29561
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:48.488500 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.476304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:34051 (local address 127.6.152.190:40293)
0806 12:29:48.476451 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.476456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.477285 (+ 829us) server_negotiation.cc:407] Connection header received
0806 12:29:48.478682 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.478685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.478731 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.478802 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:48.480155 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.480763 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.481916 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.482325 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.482470 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.484933 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:48.484937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:48.484942 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:48.484971 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:48.486399 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.486932 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.486941 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.486942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.487033 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.487431 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.487434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.487436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.487623 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:48.487732 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.488201 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.488330 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:29:48.489310 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.476701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40293 (local address 127.6.152.129:34051)
0806 12:29:48.477109 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:48.477168 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:48.478446 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:48.478996 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:48.479004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:48.479442 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:48.480002 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.480019 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.480888 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.480893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.481773 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.481783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.482493 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.482497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.482679 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.483319 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:48.483335 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:48.484769 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:48.486544 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:48.486550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:48.486563 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:48.486818 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:48.487177 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:48.487183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:48.487188 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:48.487326 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:48.487732 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:48.487738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:48.487963 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 12:29:48.488141 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 12:29:48.488440 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
I20260806 12:29:48.490516 17742 heartbeater.cc:347] Connected to a master server at 127.6.152.190:40293
I20260806 12:29:48.490751 17742 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:48.491261 17742 heartbeater.cc:511] Master 127.6.152.190:40293 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:29:48.493043 17544 ts_manager.cc:194] Registered new tserver with Master: 5964f2b36f2c41d1a1fdf54745995c05 (127.6.152.129:44457)
I20260806 12:29:48.494395 17544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:34051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:29:48.508011 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40293
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:48.615833 17747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:48.616082 17747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:48.616124 17747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:48.620013 17747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:48.620064 17747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:48.620119 17747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:29:48.625089 17747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:48.625972 17747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:48.626883 17747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:48.636052 17753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:48.636059 17755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:48.636065 17752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:48.636238 17747 server_base.cc:1034] running on GCE node
I20260806 12:29:48.636672 17747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:48.637291 17747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:48.638453 17747 hybrid_clock.cc:648] HybridClock initialized: now 1786019388638434 us; error 30 us; skew 500 ppm
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:48.641337 17747 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:29:48.642192 17747 webserver.cc:460] Webserver started at http://127.6.152.130:39969/ using document root <none> and password file <none>
I20260806 12:29:48.642721 17747 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:48.642817 17747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:48.643051 17747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:48.645023 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "1615ecdd977b400994ca06dac04d2ffa"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "57443d093f2ef012f58ebc94854cfd3c"
server_key_iv: "14c5ca5acbbd74d7ffba01221342ce0e"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.645586 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "1615ecdd977b400994ca06dac04d2ffa"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "57443d093f2ef012f58ebc94854cfd3c"
server_key_iv: "14c5ca5acbbd74d7ffba01221342ce0e"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.649151 17747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 12:29:48.651511 17764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.652707 17747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:29:48.652839 17747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "1615ecdd977b400994ca06dac04d2ffa"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "57443d093f2ef012f58ebc94854cfd3c"
server_key_iv: "14c5ca5acbbd74d7ffba01221342ce0e"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.652953 17747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:48.680949 17747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:48.683761 17747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:48.683987 17747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:48.684561 17747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:48.685365 17747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:48.685436 17747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.685509 17747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:48.685563 17747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.694329 17747 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:42467
I20260806 12:29:48.694370 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:42467 every 8 connection(s)
I20260806 12:29:48.695266 17747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:29:48.703765 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17747
I20260806 12:29:48.703922 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:48.704276 6754 external_mini_cluster.cc:1428] Setting key 7d6e17231504da38dfa496beaf66d716
I20260806 12:29:48.708655 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.696933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:41245 (local address 127.6.152.190:40293)
0806 12:29:48.697102 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.697108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.697864 (+ 756us) server_negotiation.cc:407] Connection header received
0806 12:29:48.699162 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.699166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.699228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.699303 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:48.700358 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.700873 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.701906 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.702127 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.702203 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.704777 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:48.704783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:48.704787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:48.704816 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:48.706309 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.706924 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.706926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.706927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.706998 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.707421 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.707424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.707425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.707591 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:48.707667 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.708301 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.708450 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:29:48.709262 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.697298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40293 (local address 127.6.152.130:41245)
0806 12:29:48.697714 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:48.697757 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:48.698936 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:48.699447 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:48.699453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:48.699775 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:48.700224 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.700237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.700976 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.700978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.701800 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.701807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.702250 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.702252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.702381 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.702854 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:48.702868 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:48.704598 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:48.706430 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:48.706439 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:48.706455 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:48.706819 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:48.707089 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:48.707094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:48.707100 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:48.707315 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:48.707697 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:48.707706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:48.708021 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 12:29:48.708206 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 12:29:48.708443 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":112,"threads_started":1}
I20260806 12:29:48.710400 17883 heartbeater.cc:347] Connected to a master server at 127.6.152.190:40293
I20260806 12:29:48.710671 17883 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:48.711226 17883 heartbeater.cc:511] Master 127.6.152.190:40293 requested a full tablet report, sending...
I20260806 12:29:48.712277 17544 ts_manager.cc:194] Registered new tserver with Master: 1615ecdd977b400994ca06dac04d2ffa (127.6.152.130:42467)
I20260806 12:29:48.712834 17544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:41245
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:29:48.732616 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40293
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:29:48.840309 17889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:29:48.840569 17889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:29:48.840610 17889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:29:48.844667 17889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:29:48.844717 17889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:29:48.844774 17889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:29:48.849799 17889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:40293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:29:48.850706 17889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:29:48.851719 17889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:29:48.861090 17895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:29:48.861035 17897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:48.861269 17889 server_base.cc:1034] running on GCE node
W20260806 12:29:48.861356 17894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:29:48.861724 17889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:29:48.862313 17889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:29:48.863468 17889 hybrid_clock.cc:648] HybridClock initialized: now 1786019388863441 us; error 39 us; skew 500 ppm
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:29:48.866204 17889 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:29:48.866990 17889 webserver.cc:460] Webserver started at http://127.6.152.131:34095/ using document root <none> and password file <none>
I20260806 12:29:48.867576 17889 fs_manager.cc:359] Metadata directory not provided
I20260806 12:29:48.867669 17889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:29:48.867898 17889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:29:48.869896 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "7c4cd64b7ee74377b0c75e5b61058837"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "f57a6ccbc64c42f2871e695410183be0"
server_key_iv: "889c323585422467a844c0eaa5235f32"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.870471 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "7c4cd64b7ee74377b0c75e5b61058837"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "f57a6ccbc64c42f2871e695410183be0"
server_key_iv: "889c323585422467a844c0eaa5235f32"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.874373 17889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 12:29:48.876665 17904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.877792 17889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:29:48.877923 17889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "7c4cd64b7ee74377b0c75e5b61058837"
format_stamp: "Formatted at 2026-08-06 12:29:48 on dist-test-slave-1xrh"
server_key: "f57a6ccbc64c42f2871e695410183be0"
server_key_iv: "889c323585422467a844c0eaa5235f32"
server_key_version: "encryptionkey@0"
I20260806 12:29:48.878038 17889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:29:48.903102 17889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:29:48.906452 17889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:29:48.906675 17889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:29:48.907303 17889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:29:48.908105 17889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:29:48.908177 17889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.908241 17889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:29:48.908286 17889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:29:48.916931 17889 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:40343
I20260806 12:29:48.916962 18022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:40343 every 8 connection(s)
I20260806 12:29:48.917804 17889 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:29:48.919068 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 17889
I20260806 12:29:48.919219 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:29:48.919520 6754 external_mini_cluster.cc:1428] Setting key df5046e1ec6668d8ad34437e3a3211ca
Aug 06 12:29:48 dist-test-slave-1xrh krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019388, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:29:48.930735 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.919752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:56895 (local address 127.6.152.190:40293)
0806 12:29:48.919911 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.919916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.920812 (+ 896us) server_negotiation.cc:407] Connection header received
0806 12:29:48.922204 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.922207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.922252 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.922318 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:48.923487 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.924020 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.925081 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.925286 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.925359 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.927823 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:48.927825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:48.927828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:48.927849 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:48.929061 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.929509 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.929511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.929513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.929584 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.929879 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.929882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.929883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.930052 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:48.930127 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.930530 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.930621 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:29:48.931571 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.920135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:40293 (local address 127.6.152.131:56895)
0806 12:29:48.920678 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:48.920728 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:48.922011 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:48.922478 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:48.922485 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:48.922840 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:29:48.923361 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.923377 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.924126 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.924128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.924972 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.924981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.925406 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.925409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.925555 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.926147 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:29:48.926163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:29:48.927676 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:29:48.929163 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:48.929168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:48.929179 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:48.929423 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:48.929662 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:29:48.929664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:29:48.929665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:29:48.929789 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:29:48.930137 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:29:48.930142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:29:48.930340 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 12:29:48.930484 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 12:29:48.930715 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":178,"threads_started":1}
I20260806 12:29:48.932668 18023 heartbeater.cc:347] Connected to a master server at 127.6.152.190:40293
I20260806 12:29:48.932925 18023 heartbeater.cc:464] Registering TS with master...
I20260806 12:29:48.933470 18023 heartbeater.cc:511] Master 127.6.152.190:40293 requested a full tablet report, sending...
I20260806 12:29:48.934446 17543 ts_manager.cc:194] Registered new tserver with Master: 7c4cd64b7ee74377b0c75e5b61058837 (127.6.152.131:40343)
I20260806 12:29:48.934975 17543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:56895
I20260806 12:29:48.944892 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:29:48.953989 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.946780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.6.152.190:40293)
0806 12:29:48.946925 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.946929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.947081 (+ 152us) server_negotiation.cc:407] Connection header received
0806 12:29:48.947229 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.947233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.947279 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.947359 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:29:48.948009 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.948484 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.949283 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.949513 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.949616 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.950915 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:29:48.950921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:29:48.950928 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:29:48.950950 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:29:48.952226 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.952645 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.952648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.952649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.952719 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:29:48.953011 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:29:48.953014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:29:48.953016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:29:48.953279 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:29:48.953418 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.953711 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.953821 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:29:48.956817 17543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:29:48.959798 17543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:29:48.973647 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.967835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42467 (local address 127.0.0.1:56268)
0806 12:29:48.968209 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:48.968262 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:48.968389 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:48.969173 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:48.969181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:48.969209 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:48.969370 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.969380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.970711 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.970716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.971995 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.972006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.973062 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.973066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.973190 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.973233 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 12:29:48.973343 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 12:29:48.973477 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":29,"thread_start_us":97,"threads_started":1}
I20260806 12:29:48.973654 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.968028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.6.152.130:42467)
0806 12:29:48.968381 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.968395 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.968419 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:29:48.968578 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.968585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.968790 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.968966 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:48.969511 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.970588 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.972130 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.972935 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.973055 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.973211 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.973405 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.973497 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":131,"threads_started":1}
I20260806 12:29:48.973986 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.968932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:40343 (local address 127.0.0.1:33766)
0806 12:29:48.969338 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:48.969352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:48.969451 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:48.970156 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:48.970159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:48.970178 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:48.970271 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.970277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.971680 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.971683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.972629 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.972638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.973609 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.973612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.973703 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.973731 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 12:29:48.973785 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 12:29:48.973856 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":69,"thread_start_us":96,"threads_started":1}
I20260806 12:29:48.974581 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.968600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:44457 (local address 127.0.0.1:55956)
0806 12:29:48.969306 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:48.969321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:48.969422 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:48.969943 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:48.969946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:48.969963 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:48.970116 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.970124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.971614 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.971619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.972929 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:48.972939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.974190 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.974194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:48.974287 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.974308 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 12:29:48.974378 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 12:29:48.974443 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":121,"thread_start_us":96,"threads_started":1}
I20260806 12:29:48.975001 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.968786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.6.152.129:44457)
0806 12:29:48.969171 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.969186 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.969524 (+ 338us) server_negotiation.cc:407] Connection header received
0806 12:29:48.969603 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.969607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.969790 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.969941 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:48.970247 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.971451 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.973083 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.973890 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.973991 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.974126 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.974750 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.974841 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":113,"threads_started":1}
I20260806 12:29:48.975334 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:48.969300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.6.152.131:40343)
0806 12:29:48.969703 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:48.969712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:48.969728 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:29:48.969785 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:48.969789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:48.969957 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:48.970157 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:48.970392 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.971524 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.972747 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:48.973482 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:48.974561 (+ 1079us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:48.974886 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:48.974954 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 12:29:48.975213 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":144,"threads_started":1}
I20260806 12:29:48.976747 17813 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab376d8f40f416d90904b4d7d934bc6 (DEFAULT_TABLE table=test-table [id=06487522621e4e7ab2273fa59e7e06c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:48.977726 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab376d8f40f416d90904b4d7d934bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:48.977470 17672 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab376d8f40f416d90904b4d7d934bc6 (DEFAULT_TABLE table=test-table [id=06487522621e4e7ab2273fa59e7e06c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:48.978477 17672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab376d8f40f416d90904b4d7d934bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:48.978204 17954 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab376d8f40f416d90904b4d7d934bc6 (DEFAULT_TABLE table=test-table [id=06487522621e4e7ab2273fa59e7e06c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:29:48.979192 17954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab376d8f40f416d90904b4d7d934bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:29:48.984731 18041 tablet_bootstrap.cc:492] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Bootstrap starting.
I20260806 12:29:48.986274 18042 tablet_bootstrap.cc:492] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05: Bootstrap starting.
I20260806 12:29:48.986636 18043 tablet_bootstrap.cc:492] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Bootstrap starting.
I20260806 12:29:48.987434 18041 tablet_bootstrap.cc:654] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:48.988327 18041 log.cc:824] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:48.988379 18042 tablet_bootstrap.cc:654] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:48.988807 18043 tablet_bootstrap.cc:654] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Neither blocks nor log segments found. Creating new log.
I20260806 12:29:48.989641 18042 log.cc:824] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:48.990003 18043 log.cc:824] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Log is configured to *not* fsync() on all Append() calls
I20260806 12:29:48.991616 18042 tablet_bootstrap.cc:492] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05: No bootstrap required, opened a new log
I20260806 12:29:48.991755 18041 tablet_bootstrap.cc:492] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: No bootstrap required, opened a new log
I20260806 12:29:48.991866 18042 ts_tablet_manager.cc:1397] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:29:48.991951 18041 ts_tablet_manager.cc:1397] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 12:29:48.992697 18043 tablet_bootstrap.cc:492] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: No bootstrap required, opened a new log
I20260806 12:29:48.992870 18043 ts_tablet_manager.cc:1397] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:29:48.995219 18042 raft_consensus.cc:348] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:48.995452 18042 raft_consensus.cc:374] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:48.995505 18042 raft_consensus.cc:729] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5964f2b36f2c41d1a1fdf54745995c05, State: Initialized, Role: FOLLOWER
I20260806 12:29:48.995762 18041 raft_consensus.cc:348] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:48.996006 18041 raft_consensus.cc:374] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:48.995935 18042 consensus_queue.cc:260] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:48.996075 18041 raft_consensus.cc:729] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1615ecdd977b400994ca06dac04d2ffa, State: Initialized, Role: FOLLOWER
I20260806 12:29:48.996516 18041 consensus_queue.cc:260] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:48.996610 17742 heartbeater.cc:503] Master 127.6.152.190:40293 was elected leader, sending a full tablet report...
I20260806 12:29:48.996683 18043 raft_consensus.cc:348] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:48.996918 18043 raft_consensus.cc:374] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:29:48.996978 18043 raft_consensus.cc:729] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4cd64b7ee74377b0c75e5b61058837, State: Initialized, Role: FOLLOWER
I20260806 12:29:48.997259 17883 heartbeater.cc:503] Master 127.6.152.190:40293 was elected leader, sending a full tablet report...
I20260806 12:29:48.997381 18043 consensus_queue.cc:260] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:48.998181 18042 ts_tablet_manager.cc:1428] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:29:48.998255 18043 ts_tablet_manager.cc:1428] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:29:48.998301 18041 ts_tablet_manager.cc:1428] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:29:48.998569 18023 heartbeater.cc:503] Master 127.6.152.190:40293 was elected leader, sending a full tablet report...
I20260806 12:29:49.132716 18047 raft_consensus.cc:482] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:49.132982 18047 raft_consensus.cc:504] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:49.134248 18047 leader_election.cc:288] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1615ecdd977b400994ca06dac04d2ffa (127.6.152.130:42467), 7c4cd64b7ee74377b0c75e5b61058837 (127.6.152.131:40343)
I20260806 12:29:49.138574 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.134612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:34849 (local address 127.6.152.130:42467)
0806 12:29:49.134752 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:49.134756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:49.134941 (+ 185us) server_negotiation.cc:407] Connection header received
0806 12:29:49.135099 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:49.135102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:49.135205 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:49.135299 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:49.135653 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.136319 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.137516 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.137966 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.138059 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.138134 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:49.138368 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 12:29:49.138442 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:29:49.138622 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.135105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:47985 (local address 127.6.152.131:40343)
0806 12:29:49.135266 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:49.135273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:49.135495 (+ 222us) server_negotiation.cc:407] Connection header received
0806 12:29:49.135580 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:49.135585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:49.135640 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:49.135708 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:49.135991 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.136522 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.137626 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.138005 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.138152 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.138205 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:49.138418 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 12:29:49.138485 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:29:49.138643 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.134477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42467 (local address 127.6.152.129:34849)
0806 12:29:49.134817 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:49.134834 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:49.134963 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:49.135344 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:49.135347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:49.135365 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:49.135506 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.135515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.136429 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.136432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.137381 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.137391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.138107 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.138113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.138208 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.138239 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 12:29:49.138325 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 12:29:49.138410 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":33,"thread_start_us":156,"threads_started":1}
I20260806 12:29:49.138652 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.134959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:40343 (local address 127.6.152.129:47985)
0806 12:29:49.135377 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:49.135393 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:49.135501 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:49.135744 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:49.135747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:49.135764 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:49.135880 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.135887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.136653 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.136656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.137503 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.137514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.138171 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.138179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.138263 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.138280 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 12:29:49.138386 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:29:49.138455 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":18,"thread_start_us":103,"threads_started":1}
I20260806 12:29:49.139356 17975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab376d8f40f416d90904b4d7d934bc6" candidate_uuid: "5964f2b36f2c41d1a1fdf54745995c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4cd64b7ee74377b0c75e5b61058837" is_pre_election: true
I20260806 12:29:49.139679 17975 raft_consensus.cc:2454] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5964f2b36f2c41d1a1fdf54745995c05 in term 0.
I20260806 12:29:49.139660 17834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab376d8f40f416d90904b4d7d934bc6" candidate_uuid: "5964f2b36f2c41d1a1fdf54745995c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1615ecdd977b400994ca06dac04d2ffa" is_pre_election: true
I20260806 12:29:49.139983 17834 raft_consensus.cc:2454] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5964f2b36f2c41d1a1fdf54745995c05 in term 0.
I20260806 12:29:49.140125 18048 raft_consensus.cc:482] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:29:49.140218 17627 leader_election.cc:302] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5964f2b36f2c41d1a1fdf54745995c05, 7c4cd64b7ee74377b0c75e5b61058837; no voters:
I20260806 12:29:49.140239 18048 raft_consensus.cc:504] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:49.140517 18047 raft_consensus.cc:2781] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:29:49.140635 18047 raft_consensus.cc:482] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:29:49.140682 18047 raft_consensus.cc:3037] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:49.141399 18048 leader_election.cc:288] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5964f2b36f2c41d1a1fdf54745995c05 (127.6.152.129:44457), 7c4cd64b7ee74377b0c75e5b61058837 (127.6.152.131:40343)
I20260806 12:29:49.142100 18047 raft_consensus.cc:504] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:49.142525 18047 leader_election.cc:288] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [CANDIDATE]: Term 1 election: Requested vote from peers 1615ecdd977b400994ca06dac04d2ffa (127.6.152.130:42467), 7c4cd64b7ee74377b0c75e5b61058837 (127.6.152.131:40343)
I20260806 12:29:49.142905 17834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab376d8f40f416d90904b4d7d934bc6" candidate_uuid: "5964f2b36f2c41d1a1fdf54745995c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1615ecdd977b400994ca06dac04d2ffa"
I20260806 12:29:49.143026 17834 raft_consensus.cc:3037] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:49.143054 17975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab376d8f40f416d90904b4d7d934bc6" candidate_uuid: "5964f2b36f2c41d1a1fdf54745995c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4cd64b7ee74377b0c75e5b61058837"
I20260806 12:29:49.143200 17975 raft_consensus.cc:3037] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:29:49.144167 17834 raft_consensus.cc:2454] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5964f2b36f2c41d1a1fdf54745995c05 in term 1.
I20260806 12:29:49.144443 17627 leader_election.cc:302] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1615ecdd977b400994ca06dac04d2ffa, 5964f2b36f2c41d1a1fdf54745995c05; no voters:
I20260806 12:29:49.144521 17975 raft_consensus.cc:2454] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5964f2b36f2c41d1a1fdf54745995c05 in term 1.
I20260806 12:29:49.144639 18047 raft_consensus.cc:2781] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:29:49.144856 18047 raft_consensus.cc:686] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [term 1 LEADER]: Becoming Leader. State: Replica: 5964f2b36f2c41d1a1fdf54745995c05, State: Running, Role: LEADER
I20260806 12:29:49.145233 18047 consensus_queue.cc:237] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } }
I20260806 12:29:49.146845 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.141924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:53983 (local address 127.6.152.131:40343)
0806 12:29:49.142326 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:49.142329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:49.142631 (+ 302us) server_negotiation.cc:407] Connection header received
0806 12:29:49.142805 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:49.142808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:49.142855 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:49.142928 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:49.143328 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.143988 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.145584 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.146154 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.146231 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.146273 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:49.146653 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 12:29:49.146719 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260806 12:29:49.147047 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.141576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:44457 (local address 127.6.152.130:33709)
0806 12:29:49.141678 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:49.141713 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:49.141783 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:49.142203 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:49.142206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:49.142228 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:49.142334 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.142340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.144714 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.144717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.145793 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.145802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.146627 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.146634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.146748 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.146773 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 12:29:49.146853 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 12:29:49.146924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:29:49.147246 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.141836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:40343 (local address 127.6.152.130:53983)
0806 12:29:49.142537 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:49.142551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:29:49.142712 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:29:49.143022 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:29:49.143025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:29:49.143042 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:29:49.143175 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.143182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.144135 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.144137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.145445 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:29:49.145455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.146385 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.146389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:29:49.146509 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.146543 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:29:49.146960 (+ 417us) client_negotiation.cc:240] Negotiation successful
0806 12:29:49.147034 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":95,"threads_started":1}
I20260806 12:29:49.147946 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.141638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:33709 (local address 127.6.152.129:44457)
0806 12:29:49.141939 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:49.141945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:49.141961 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:29:49.142001 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:49.142003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:49.142072 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:49.142150 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:29:49.142450 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.143313 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.145931 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.146472 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.146617 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.146664 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:49.147749 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0806 12:29:49.147845 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260806 12:29:49.148015 17975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab376d8f40f416d90904b4d7d934bc6" candidate_uuid: "1615ecdd977b400994ca06dac04d2ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4cd64b7ee74377b0c75e5b61058837" is_pre_election: true
I20260806 12:29:49.148178 17975 raft_consensus.cc:2379] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1615ecdd977b400994ca06dac04d2ffa in current term 1: Already voted for candidate 5964f2b36f2c41d1a1fdf54745995c05 in this term.
I20260806 12:29:49.148622 17694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab376d8f40f416d90904b4d7d934bc6" candidate_uuid: "1615ecdd977b400994ca06dac04d2ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5964f2b36f2c41d1a1fdf54745995c05" is_pre_election: true
I20260806 12:29:49.149163 17767 leader_election.cc:302] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1615ecdd977b400994ca06dac04d2ffa; no voters: 5964f2b36f2c41d1a1fdf54745995c05, 7c4cd64b7ee74377b0c75e5b61058837
I20260806 12:29:49.148983 17543 catalog_manager.cc:5495] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5964f2b36f2c41d1a1fdf54745995c05 (127.6.152.129). New cstate: current_term: 1 leader_uuid: "5964f2b36f2c41d1a1fdf54745995c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5964f2b36f2c41d1a1fdf54745995c05" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 44457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260806 12:29:49.149392 18048 raft_consensus.cc:2726] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 12:29:49.165457 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.162228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.6.152.129:44457)
0806 12:29:49.162350 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 12:29:49.162354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:29:49.162367 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:29:49.162512 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:29:49.162514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:29:49.162563 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:29:49.162659 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:29:49.162872 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.163416 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.164222 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:29:49.164418 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:29:49.164490 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:29:49.164740 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:29:49.165120 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:29:49.165208 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:29:49.165263 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 12:29:49.165302 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 12:29:49.168735 18024 tablet.cc:2369] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:29:49.170954 17975 raft_consensus.cc:1261] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 1 FOLLOWER]: Refusing update from remote peer 5964f2b36f2c41d1a1fdf54745995c05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:49.171022 17834 raft_consensus.cc:1261] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 1 FOLLOWER]: Refusing update from remote peer 5964f2b36f2c41d1a1fdf54745995c05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:29:49.171685 18053 consensus_queue.cc:1035] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c4cd64b7ee74377b0c75e5b61058837" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:49.171872 18047 consensus_queue.cc:1035] T 7ab376d8f40f416d90904b4d7d934bc6 P 5964f2b36f2c41d1a1fdf54745995c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1615ecdd977b400994ca06dac04d2ffa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:29:49.178333 18056 mvcc.cc:205] Tried to move back new op lower bound from 7315535418037800960 to 7315535417941544960. Current Snapshot: MvccSnapshot[applied={T|T < 7315535418037800960}]
I20260806 12:29:49.178838 18058 mvcc.cc:205] Tried to move back new op lower bound from 7315535418037800960 to 7315535417941544960. Current Snapshot: MvccSnapshot[applied={T|T < 7315535418037800960}]
I20260806 12:29:49.179803 18057 mvcc.cc:205] Tried to move back new op lower bound from 7315535418037800960 to 7315535417941544960. Current Snapshot: MvccSnapshot[applied={T|T < 7315535418037800960}]
I20260806 12:29:49.187182 17544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:49.187352 17544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:29:49.189580 17544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2be40d7e58e542bf99c8d3d949b8f786: Sending DeleteTablet for 3 replicas of tablet 7ab376d8f40f416d90904b4d7d934bc6
I20260806 12:29:49.190274 17813 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab376d8f40f416d90904b4d7d934bc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:49 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:56268
I20260806 12:29:49.190471 17954 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab376d8f40f416d90904b4d7d934bc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:49 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:33766
I20260806 12:29:49.190474 17672 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab376d8f40f416d90904b4d7d934bc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:29:49 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:55956
I20260806 12:29:49.190886 18065 tablet_replica.cc:330] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: stopping tablet replica
I20260806 12:29:49.190883 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17606
I20260806 12:29:49.191917 18067 tablet_replica.cc:330] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: stopping tablet replica
I20260806 12:29:49.192158 18067 raft_consensus.cc:2229] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:49.192144 18065 raft_consensus.cc:2229] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:29:49.192425 18067 raft_consensus.cc:2258] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:49.193310 18065 raft_consensus.cc:2258] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:29:49.194484 18067 ts_tablet_manager.cc:1905] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:49.194595 18065 ts_tablet_manager.cc:1905] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:29:49.197202 18067 ts_tablet_manager.cc:1918] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:49.197202 18065 ts_tablet_manager.cc:1918] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:29:49.197304 18065 log.cc:1196] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/7ab376d8f40f416d90904b4d7d934bc6
I20260806 12:29:49.197304 18067 log.cc:1196] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/7ab376d8f40f416d90904b4d7d934bc6
I20260806 12:29:49.197671 18067 ts_tablet_manager.cc:1939] T 7ab376d8f40f416d90904b4d7d934bc6 P 7c4cd64b7ee74377b0c75e5b61058837: Deleting consensus metadata
I20260806 12:29:49.197681 18065 ts_tablet_manager.cc:1939] T 7ab376d8f40f416d90904b4d7d934bc6 P 1615ecdd977b400994ca06dac04d2ffa: Deleting consensus metadata
I20260806 12:29:49.198835 17527 catalog_manager.cc:4841] TS 1615ecdd977b400994ca06dac04d2ffa (127.6.152.130:42467): tablet 7ab376d8f40f416d90904b4d7d934bc6 (table test-table [id=06487522621e4e7ab2273fa59e7e06c2]) successfully deleted
I20260806 12:29:49.199061 17527 catalog_manager.cc:4841] TS 7c4cd64b7ee74377b0c75e5b61058837 (127.6.152.131:40343): tablet 7ab376d8f40f416d90904b4d7d934bc6 (table test-table [id=06487522621e4e7ab2273fa59e7e06c2]) successfully deleted
W20260806 12:29:49.200416 17527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:44457 (error 108)
I20260806 12:29:49.200506 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17747
I20260806 12:29:49.202147 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:29:49.201811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:44457 (local address 127.0.0.1:55982)
0806 12:29:49.201963 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:29:49.202048 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:44457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 12:29:49.202288 17527 catalog_manager.cc:4584] TS 5964f2b36f2c41d1a1fdf54745995c05 (127.6.152.129:44457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ab376d8f40f416d90904b4d7d934bc6: Network error: Client connection negotiation failed: client connection to 127.6.152.129:44457: connect: Connection refused (error 111)
I20260806 12:29:49.207043 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17889
I20260806 12:29:49.213260 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 17509
2026-08-06T12:29:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2402 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:29:49 dist-test-slave-1xrh krb5kdc[18071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:29:49 dist-test-slave-1xrh krb5kdc[18071](info): set up 2 sockets
Aug 06 12:29:49 dist-test-slave-1xrh krb5kdc[18071](info): commencing operation
krb5kdc: starting...
W20260806 12:29:50.267576 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:29:50 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:29:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/rangeradmin_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/rangeradmin_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/rangerlookup_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/rangerlookup_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
I20260806 12:29:50.441629 6754 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknpcosN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/postgres -l logfile start
2026-08-06 12:29:52.943 UTC [18102] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 12:29:52.943 UTC [18102] LOG: listening on IPv4 address "127.6.152.148", port 45793
2026-08-06 12:29:52.946 UTC [18102] LOG: listening on Unix socket "/tmp/.s.PGSQL.45793"
2026-08-06 12:29:52.967 UTC [18105] LOG: database system was shut down at 2026-08-06 12:29:51 UTC
2026-08-06 12:29:52.971 UTC [18102] LOG: database system is ready to accept connections
I20260806 12:29:53.947539 6754 mini_postgres.cc:96] Postgres bound to 45793
2026-08-06 12:29:53.955 UTC [18113] FATAL: database "slave" does not exist
127.6.152.148:45793 - accepting connections
I20260806 12:29:53.956163 6754 mini_ranger.cc:161] Starting Ranger...
I20260806 12:29:53.971467 6754 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 12:29:54.412739 6754 mini_ranger.cc:87] Created ranger Postgres database
I20260806 12:29:54.412827 6754 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-admin
2026-08-06 12:29:54,728 [I] DB FLAVOR :POSTGRES
2026-08-06 12:29:54,728 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:29:54,728 [I] Checking connection..
2026-08-06 12:29:54,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 12:29:54,992 [I] Checking connection passed.
2026-08-06 12:29:54,993 [I] --------- Verifying version history table ---------
2026-08-06 12:29:54,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:29:55,231 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:29:55,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:29:55,504 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:29:55,504 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 12:29:55,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 12:29:55.728 UTC [18181] WARNING: there is no transaction in progress
2026-08-06 12:29:55,734 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 12:29:55,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:29:55,978 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 12:29:55,978 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:29:55,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 12:29:56,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 12:29:56,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 12:29:56,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:29:56,972 [I] Table x_portal_user does not exist in database ranger
2026-08-06 12:29:56,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:29:57,215 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 12:29:57,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:29:57,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:29:57,704 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 12:29:57,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 12:29:58.617 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.633 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.648 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.663 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.678 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.694 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.746 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.753 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.767 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.776 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.786 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.800 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.809 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.815 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.822 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:58.828 UTC [18298] WARNING: there is no transaction in progress
2026-08-06 12:29:59,454 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 12:29:59,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 12:29:59,673 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 12:29:59,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:29:59,902 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 12:29:59,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:30:00,131 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 12:30:00,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:30:00,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:30:00,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 12:30:00,851 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 12:30:00,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 12:30:01,075 [I] DB_PATCHES have already been applied
I20260806 12:30:01.081648 6754 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknpcosN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 12:30:01.081737 6754 mini_ranger.cc:193] Using host: 127.6.152.148
I20260806 12:30:01.085858 6754 mini_ranger.cc:227] Ranger admin URL: http://127.6.152.148:33385
Aug 06, 2026 12:30:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:30:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:30:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:30:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:30:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:30:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:30:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 12:30:02.243160 6754 mini_ranger.cc:160] Time spent starting Ranger: real 8.287s user 0.000s sys 0.011s
Aug 06, 2026 12:30:24 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:30:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 12:30:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:30:24 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:30:25 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 12:30:25.657241 6754 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/rangerkms_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/rangerkms_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.152.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.152.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 12:30:25 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019425, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 12:30:25.753880 6754 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 12:30:25.766489 6754 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 12:30:26.621650 6754 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 12:30:26.621738 6754 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms
I20260806 12:30:26.621857 6754 mini_ranger_kms.cc:227] Using postgres at 127.6.152.148:45793
2026-08-06 12:30:27,129 [I] DB FLAVOR :POSTGRES
2026-08-06 12:30:27,129 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:30:27,129 [I] Checking connection
2026-08-06 12:30:27,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 12:30:27,549 [I] connection success
2026-08-06 12:30:27,549 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 12:30:27,549 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 12:30:27,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 12:30:27,902 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 12:30:27,902 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:30:27,902 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 12:30:27,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:45793/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 12:30:28,175 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 12:30:28.678231 6754 mini_ranger_kms.cc:310] Created kms service
I20260806 12:30:28.872148 6754 mini_ranger_kms.cc:326] Created kudu user
I20260806 12:30:28.914463 6754 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 12:30:28 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019425, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:29.304692 6754 mini_ranger_kms.cc:384] Added ranger policy
I20260806 12:30:29.304872 6754 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 12:30:29.304914 6754 mini_ranger_kms.cc:240] Using host: 127.6.152.148
I20260806 12:30:29.309093 6754 mini_ranger_kms.cc:276] Ranger KMS PID: 18488
I20260806 12:30:29.309235 6754 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.152.148:59413
Aug 06, 2026 12:30:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:30:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:30:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:30:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:30:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:30:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 12:30:31.700415 6754 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.947s user 0.001s sys 0.482s
I20260806 12:30:31.700568 6754 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 12:30:31.700691 6754 mini_ranger_kms.cc:401] 127.6.152.148:59413/kms/v1/keys
Aug 06 12:30:32 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786019432, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.152.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 12:30:32 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019432, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.152.148@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
Aug 06 12:30:32 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019432, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.152.148@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
Aug 06 12:30:35 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:30:35.402614 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:43151
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:43151
--ranger_config_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:59413/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:30:35.550427 18539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:30:35.550665 18539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:30:35.550740 18539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:30:35.554925 18539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:30:35.554980 18539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:30:35.555027 18539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:30:35.555059 18539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:30:35.555099 18539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:30:35.560871 18539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.152.148:59413/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:43151
--ranger_config_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:43151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:30:35.561856 18539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:30:35.562858 18539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:30:35.572587 18544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:35.572688 18545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:35.572778 18547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:30:35.572819 18539 server_base.cc:1034] running on GCE node
I20260806 12:30:35.573403 18539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:30:35.574470 18539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:30:35.575685 18539 hybrid_clock.cc:648] HybridClock initialized: now 1786019435575636 us; error 64 us; skew 500 ppm
Aug 06 12:30:35 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:30:35.578982 18539 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:30:35.579962 18539 webserver.cc:460] Webserver started at http://127.6.152.190:41685/ using document root <none> and password file <none>
I20260806 12:30:35.580494 18539 fs_manager.cc:359] Metadata directory not provided
I20260806 12:30:35.580575 18539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:30:35.580817 18539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:30:35 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019435, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:35.645068 18539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "6420f7974ef648b28c7248038e1ddf77"
format_stamp: "Formatted at 2026-08-06 12:30:35 on dist-test-slave-1xrh"
server_key: "fc2cb7c6e62131827dc545ee9b041ad7"
server_key_iv: "ae6a1e4050e7d6806033180b63c9a554"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:35.645713 18539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "6420f7974ef648b28c7248038e1ddf77"
format_stamp: "Formatted at 2026-08-06 12:30:35 on dist-test-slave-1xrh"
server_key: "fc2cb7c6e62131827dc545ee9b041ad7"
server_key_iv: "ae6a1e4050e7d6806033180b63c9a554"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:35.649974 18539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 12:30:35.668144 18555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:35.669962 18539 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 12:30:35.670126 18539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "6420f7974ef648b28c7248038e1ddf77"
format_stamp: "Formatted at 2026-08-06 12:30:35 on dist-test-slave-1xrh"
server_key: "fc2cb7c6e62131827dc545ee9b041ad7"
server_key_iv: "ae6a1e4050e7d6806033180b63c9a554"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:35.670284 18539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:30:35.725378 18539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:30:35.728639 18539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:30:35.728904 18539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:30:35.740177 18539 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:43151
I20260806 12:30:35.740573 18610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:43151 every 8 connection(s)
I20260806 12:30:35.741426 18539 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:30:35.741813 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 18539
I20260806 12:30:35.741932 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:30:35.742239 6754 external_mini_cluster.cc:1428] Setting key d6069deccc0b1ba857ef6fc4b12e30fd
I20260806 12:30:35.746212 18611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:30:35.754997 18611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77: Bootstrap starting.
Aug 06 12:30:35 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:30:35.758224 18611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77: Neither blocks nor log segments found. Creating new log.
I20260806 12:30:35.760317 18611 log.cc:824] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77: Log is configured to *not* fsync() on all Append() calls
I20260806 12:30:35.761302 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:35.743850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.6.152.190:43151)
0806 12:30:35.746621 (+ 2771us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:35.746655 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:35.746988 (+ 333us) server_negotiation.cc:407] Connection header received
0806 12:30:35.748178 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:35.748198 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:35.748539 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:35.748874 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:30:35.749941 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:35.751301 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:35.752418 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:35.752946 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:35.753117 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:35.756006 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:30:35.756011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:30:35.756024 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:30:35.756051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:30:35.758019 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:35.758566 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:35.758571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:35.758576 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:35.758676 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:35.758962 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:35.758967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:35.758969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:35.759401 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:30:35.759623 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:35.759916 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 12:30:35.760306 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2629,"thread_start_us":220,"threads_started":1}
I20260806 12:30:35.762844 18611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77: No bootstrap required, opened a new log
I20260806 12:30:35.768368 18611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6420f7974ef648b28c7248038e1ddf77" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43151 } }
I20260806 12:30:35.768638 18611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:30:35.768697 18611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6420f7974ef648b28c7248038e1ddf77, State: Initialized, Role: FOLLOWER
I20260806 12:30:35.769142 18611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6420f7974ef648b28c7248038e1ddf77" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43151 } }
I20260806 12:30:35.769289 18611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:30:35.769359 18611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:30:35.769448 18611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:30:35.770830 18611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6420f7974ef648b28c7248038e1ddf77" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43151 } }
I20260806 12:30:35.771229 18611 leader_election.cc:302] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6420f7974ef648b28c7248038e1ddf77; no voters:
I20260806 12:30:35.771608 18611 leader_election.cc:288] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:30:35.771893 18616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:30:35.772507 18616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [term 1 LEADER]: Becoming Leader. State: Replica: 6420f7974ef648b28c7248038e1ddf77, State: Running, Role: LEADER
I20260806 12:30:35.772979 18611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:30:35.773028 18616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6420f7974ef648b28c7248038e1ddf77" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43151 } }
I20260806 12:30:35.774447 18616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6420f7974ef648b28c7248038e1ddf77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6420f7974ef648b28c7248038e1ddf77" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43151 } } }
I20260806 12:30:35.774605 18616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [sys.catalog]: This master's current role is: LEADER
I20260806 12:30:35.774993 18616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6420f7974ef648b28c7248038e1ddf77. Latest consensus state: current_term: 1 leader_uuid: "6420f7974ef648b28c7248038e1ddf77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6420f7974ef648b28c7248038e1ddf77" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43151 } } }
I20260806 12:30:35.775115 18616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77 [sys.catalog]: This master's current role is: LEADER
I20260806 12:30:35.775390 18626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:30:35.779417 18626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:30:35.780062 18611 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 12:30:35.799250 18626 catalog_manager.cc:1347] Generated new cluster ID: 17c81b0d73884074a4e8264a26f6b269
I20260806 12:30:35.799357 18626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:30:35.817157 18626 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:30:35.818464 18626 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:30:35.825168 18626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77: Generated new TSK 0
I20260806 12:30:35.825953 18626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:30:36.766211 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:59413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43151
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:30:36.940678 18658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:30:36.940979 18658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:30:36.941025 18658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:30:36.945230 18658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:30:36.945286 18658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:30:36.945343 18658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:30:36.950953 18658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.152.148:59413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:30:36.952114 18658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:30:36.953137 18658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:30:36.965700 18658 server_base.cc:1034] running on GCE node
W20260806 12:30:36.965718 18665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:36.965610 18668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:36.965934 18664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:30:36.966372 18658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:30:36.967108 18658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:30:36.968319 18658 hybrid_clock.cc:648] HybridClock initialized: now 1786019436968277 us; error 54 us; skew 500 ppm
Aug 06 12:30:36 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019436, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:30:36.980130 18658 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:30:36.981586 18658 webserver.cc:460] Webserver started at http://127.6.152.129:43709/ using document root <none> and password file <none>
I20260806 12:30:36.982645 18658 fs_manager.cc:359] Metadata directory not provided
I20260806 12:30:36.982836 18658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:30:36.983249 18658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:30:36 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019436, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:37.004526 18658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "3aaec30a7c8e43559a862164477d63db"
format_stamp: "Formatted at 2026-08-06 12:30:36 on dist-test-slave-1xrh"
server_key: "2b29fe3a021d06815323b76a234e01e6"
server_key_iv: "360298b7c7bc13d2fad3819c91b6ef3e"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.007113 18658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "3aaec30a7c8e43559a862164477d63db"
format_stamp: "Formatted at 2026-08-06 12:30:36 on dist-test-slave-1xrh"
server_key: "2b29fe3a021d06815323b76a234e01e6"
server_key_iv: "360298b7c7bc13d2fad3819c91b6ef3e"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.012746 18658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 12:30:37.029256 18677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.031179 18658 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 12:30:37.031320 18658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "3aaec30a7c8e43559a862164477d63db"
format_stamp: "Formatted at 2026-08-06 12:30:36 on dist-test-slave-1xrh"
server_key: "2b29fe3a021d06815323b76a234e01e6"
server_key_iv: "360298b7c7bc13d2fad3819c91b6ef3e"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.031464 18658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:30:37.055760 18658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:30:37.059101 18658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:30:37.059413 18658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:30:37.060145 18658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:30:37.061213 18658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:30:37.061291 18658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.061400 18658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:30:37.061465 18658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.078125 18658 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:42377
I20260806 12:30:37.079504 18658 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:30:37.079891 18794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:42377 every 8 connection(s)
I20260806 12:30:37.084391 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 18658
I20260806 12:30:37.084513 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:30:37.084847 6754 external_mini_cluster.cc:1428] Setting key 0103d41028372cab79099d4009642bcc
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019436, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:30:37.114470 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.082597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:58569 (local address 127.6.152.190:43151)
0806 12:30:37.082983 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.082988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.098938 (+ 15950us) server_negotiation.cc:407] Connection header received
0806 12:30:37.100376 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.100380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.100442 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.100733 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:30:37.103189 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.104103 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.105879 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.106233 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.106320 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.109702 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:30:37.109709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:30:37.109713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:30:37.109741 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:30:37.111701 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.112404 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.112408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.112410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.112519 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.112985 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.112990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.112994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.113244 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:30:37.113341 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.114106 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.114254 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:30:37.116569 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.083045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:43151 (local address 127.6.152.129:58569)
0806 12:30:37.098544 (+ 15499us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:37.098627 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:37.100123 (+ 1496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:37.101741 (+ 1618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:37.101751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:37.102208 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:30:37.102975 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.102995 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.104272 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.104275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.105733 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.105744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.106565 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.106570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.106794 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.107602 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:30:37.107622 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:30:37.109500 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:30:37.111851 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:30:37.111857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:30:37.111873 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:30:37.112232 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:30:37.112654 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:30:37.112660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:30:37.112662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:30:37.112855 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:30:37.113424 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:30:37.113433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:30:37.113775 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 12:30:37.115011 (+ 1236us) client_negotiation.cc:240] Negotiation successful
0806 12:30:37.115368 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15178,"thread_start_us":168,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:30:37.118114 18795 heartbeater.cc:347] Connected to a master server at 127.6.152.190:43151
I20260806 12:30:37.118373 18795 heartbeater.cc:464] Registering TS with master...
I20260806 12:30:37.118907 18795 heartbeater.cc:511] Master 127.6.152.190:43151 requested a full tablet report, sending...
I20260806 12:30:37.120720 18573 ts_manager.cc:194] Registered new tserver with Master: 3aaec30a7c8e43559a862164477d63db (127.6.152.129:42377)
I20260806 12:30:37.121974 18573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:58569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:30:37.138690 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:59413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43151
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 12:30:37.290443 18804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:30:37.290725 18804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:30:37.290802 18804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:30:37.296777 18804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:30:37.296867 18804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:30:37.296952 18804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:30:37.304881 18804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.152.148:59413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:30:37.306191 18804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:30:37.307519 18804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:30:37.325109 18814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:37.325440 18812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:37.325451 18811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:30:37.325984 18804 server_base.cc:1034] running on GCE node
I20260806 12:30:37.326563 18804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:30:37.327418 18804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:30:37.328622 18804 hybrid_clock.cc:648] HybridClock initialized: now 1786019437328598 us; error 37 us; skew 500 ppm
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:30:37.332789 18804 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:30:37.333891 18804 webserver.cc:460] Webserver started at http://127.6.152.130:36067/ using document root <none> and password file <none>
I20260806 12:30:37.334573 18804 fs_manager.cc:359] Metadata directory not provided
I20260806 12:30:37.334666 18804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:30:37.334923 18804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:37.352298 18804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "2b26ea55a5d54bb49ab77b1b8fef2416"
format_stamp: "Formatted at 2026-08-06 12:30:37 on dist-test-slave-1xrh"
server_key: "4ab10fc59052a77f0ae99583bdcdef67"
server_key_iv: "acb5db9b04da4110ef9d22fbe5b5ba89"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.354053 18804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "2b26ea55a5d54bb49ab77b1b8fef2416"
format_stamp: "Formatted at 2026-08-06 12:30:37 on dist-test-slave-1xrh"
server_key: "4ab10fc59052a77f0ae99583bdcdef67"
server_key_iv: "acb5db9b04da4110ef9d22fbe5b5ba89"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.360618 18804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 12:30:37.390254 18821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.392098 18804 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 12:30:37.392248 18804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "2b26ea55a5d54bb49ab77b1b8fef2416"
format_stamp: "Formatted at 2026-08-06 12:30:37 on dist-test-slave-1xrh"
server_key: "4ab10fc59052a77f0ae99583bdcdef67"
server_key_iv: "acb5db9b04da4110ef9d22fbe5b5ba89"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.392402 18804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:30:37.440304 18804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:30:37.444204 18804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:30:37.444460 18804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:30:37.445395 18804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:30:37.446723 18804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:30:37.446847 18804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.446964 18804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:30:37.447039 18804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.466055 18804 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:45385
I20260806 12:30:37.467344 18804 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:30:37.469259 18936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:45385 every 8 connection(s)
I20260806 12:30:37.470520 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 18804
I20260806 12:30:37.470666 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:30:37.471025 6754 external_mini_cluster.cc:1428] Setting key 609b25efba788d5520c3bfa997e7c54d
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:30:37.493847 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.472851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:34219 (local address 127.6.152.190:43151)
0806 12:30:37.472996 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.473001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.474351 (+ 1350us) server_negotiation.cc:407] Connection header received
0806 12:30:37.478482 (+ 4131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.478488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.478550 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.478607 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:30:37.482036 (+ 3429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.482916 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.485096 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.485415 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.485479 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.489163 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:30:37.489170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:30:37.489174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:30:37.489207 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:30:37.491352 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.492027 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.492031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.492033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.492131 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.492535 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.492539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.492541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.492753 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:30:37.492865 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.493522 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.493672 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:30:37.494832 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.473431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:43151 (local address 127.6.152.130:34219)
0806 12:30:37.474108 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:37.474176 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:37.478197 (+ 4021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:37.479033 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:37.479045 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:37.479673 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:30:37.480464 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.480489 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.483097 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.483103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.484944 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.484959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.485678 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.485680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.485844 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.486566 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:30:37.486583 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:30:37.488936 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:30:37.491508 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:30:37.491515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:30:37.491534 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:30:37.491902 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:30:37.492247 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:30:37.492250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:30:37.492252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:30:37.492423 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:30:37.492950 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:30:37.492956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:30:37.493268 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 12:30:37.493465 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 12:30:37.493781 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":209,"threads_started":1}
I20260806 12:30:37.496568 18937 heartbeater.cc:347] Connected to a master server at 127.6.152.190:43151
I20260806 12:30:37.496832 18937 heartbeater.cc:464] Registering TS with master...
I20260806 12:30:37.497505 18937 heartbeater.cc:511] Master 127.6.152.190:43151 requested a full tablet report, sending...
I20260806 12:30:37.498796 18573 ts_manager.cc:194] Registered new tserver with Master: 2b26ea55a5d54bb49ab77b1b8fef2416 (127.6.152.130:45385)
I20260806 12:30:37.499626 18573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:34219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:30:37.514811 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:59413/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43151
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:30:37.689900 18942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:30:37.690299 18942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:30:37.690414 18942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:30:37.696148 18942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:30:37.696219 18942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:30:37.696295 18942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:30:37.703898 18942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.152.148:59413/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:30:37.705271 18942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:30:37.706840 18942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:30:37.720633 18951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:37.720629 18949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:30:37.720604 18947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:30:37.721889 18942 server_base.cc:1034] running on GCE node
I20260806 12:30:37.722357 18942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:30:37.722999 18942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:37.724201 18942 hybrid_clock.cc:648] HybridClock initialized: now 1786019437724173 us; error 39 us; skew 500 ppm
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:30:37.728034 18942 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:30:37.729017 18942 webserver.cc:460] Webserver started at http://127.6.152.131:40087/ using document root <none> and password file <none>
I20260806 12:30:37.729653 18942 fs_manager.cc:359] Metadata directory not provided
I20260806 12:30:37.729787 18942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:30:37.730054 18942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:37.770216 18942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "ae0bff7487fe4256bca8c579d5f562c2"
format_stamp: "Formatted at 2026-08-06 12:30:37 on dist-test-slave-1xrh"
server_key: "007b72fa395ebc1aa3856d0e655cda05"
server_key_iv: "1531f0e8ca03e417aae7f5ddd7f24620"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.771199 18942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "ae0bff7487fe4256bca8c579d5f562c2"
format_stamp: "Formatted at 2026-08-06 12:30:37 on dist-test-slave-1xrh"
server_key: "007b72fa395ebc1aa3856d0e655cda05"
server_key_iv: "1531f0e8ca03e417aae7f5ddd7f24620"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.777038 18942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 12:30:37.797044 18960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.799010 18942 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 12:30:37.799232 18942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "ae0bff7487fe4256bca8c579d5f562c2"
format_stamp: "Formatted at 2026-08-06 12:30:37 on dist-test-slave-1xrh"
server_key: "007b72fa395ebc1aa3856d0e655cda05"
server_key_iv: "1531f0e8ca03e417aae7f5ddd7f24620"
server_key_version: "kuduclusterkey@0"
I20260806 12:30:37.799455 18942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:30:37.825902 18942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:30:37.829695 18942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:30:37.829960 18942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:30:37.830699 18942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:30:37.831902 18942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:30:37.832010 18942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.832124 18942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:30:37.832194 18942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:30:37.857560 18942 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:39311
I20260806 12:30:37.858578 18942 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:30:37.859910 19076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:39311 every 8 connection(s)
I20260806 12:30:37.859948 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 18942
I20260806 12:30:37.860080 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:30:37.860396 6754 external_mini_cluster.cc:1428] Setting key 2a5158d01374963089af47244f76f02f
Aug 06 12:30:37 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:30:37.876704 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.862400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:48779 (local address 127.6.152.190:43151)
0806 12:30:37.862526 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.862532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.863105 (+ 573us) server_negotiation.cc:407] Connection header received
0806 12:30:37.864458 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.864462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.864514 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.864563 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:30:37.866210 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.867019 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.868668 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.869005 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.869078 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.872412 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:30:37.872416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:30:37.872419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:30:37.872445 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:30:37.874268 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.874930 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.874934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.874936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.875017 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.875428 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.875432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.875434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.875654 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:30:37.875733 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.876431 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.876551 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:30:37.879344 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.861327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:43151 (local address 127.6.152.131:48779)
0806 12:30:37.862905 (+ 1578us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:37.862963 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:37.864212 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:37.864969 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:37.864978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:37.865369 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:30:37.866050 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.866068 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.867196 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.867199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.868531 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.868543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.869274 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.869277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.869494 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.870273 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:30:37.870301 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:30:37.872228 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:30:37.874448 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:30:37.874454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:30:37.874469 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:30:37.874817 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:30:37.875135 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:30:37.875139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:30:37.875141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:30:37.875303 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:30:37.875864 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:30:37.875870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:30:37.876164 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 12:30:37.877148 (+ 984us) client_negotiation.cc:240] Negotiation successful
0806 12:30:37.877485 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"thread_start_us":173,"threads_started":1}
I20260806 12:30:37.880883 19077 heartbeater.cc:347] Connected to a master server at 127.6.152.190:43151
I20260806 12:30:37.881121 19077 heartbeater.cc:464] Registering TS with master...
I20260806 12:30:37.881677 19077 heartbeater.cc:511] Master 127.6.152.190:43151 requested a full tablet report, sending...
I20260806 12:30:37.882885 18573 ts_manager.cc:194] Registered new tserver with Master: ae0bff7487fe4256bca8c579d5f562c2 (127.6.152.131:39311)
I20260806 12:30:37.883684 18573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:48779
I20260806 12:30:37.886492 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:30:37.897217 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.888820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.6.152.190:43151)
0806 12:30:37.888938 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.888942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.889446 (+ 504us) server_negotiation.cc:407] Connection header received
0806 12:30:37.889604 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.889607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.889657 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.889741 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:30:37.890634 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.891231 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.892087 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.892316 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.892423 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.893840 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:30:37.893844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:30:37.893847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:30:37.893872 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:30:37.895451 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.895878 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.895882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.895884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.895965 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:30:37.896263 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:30:37.896266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:30:37.896268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:30:37.896446 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:30:37.896594 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.896846 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.896974 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:30:37.900282 18573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:30:37.905073 18573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:30:37.936421 19092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.919759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:45385 (local address 127.0.0.1:33994)
0806 12:30:37.923619 (+ 3860us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:37.923672 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:37.923881 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:37.925891 (+ 2010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:37.925897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:37.925919 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:30:37.926056 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.926063 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.931324 (+ 5261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.931328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.932695 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.932708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.935816 (+ 3108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.935820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.935939 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.935982 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 12:30:37.936095 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 12:30:37.936217 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3278,"thread_start_us":141,"threads_started":1}
I20260806 12:30:37.937568 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.921463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:39311 (local address 127.0.0.1:36154)
0806 12:30:37.924170 (+ 2707us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:37.924184 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:37.924320 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:37.926177 (+ 1857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:37.926180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:37.926196 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:30:37.926322 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.926328 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.927956 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.927960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.929618 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.929631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.937118 (+ 7487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.937122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.937248 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.937274 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:30:37.937362 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 12:30:37.937433 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2639,"thread_start_us":248,"threads_started":1}
I20260806 12:30:37.938124 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.922113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.6.152.131:39311)
0806 12:30:37.923348 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.923405 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.924420 (+ 1015us) server_negotiation.cc:407] Connection header received
0806 12:30:37.924500 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.924505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.924722 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.924935 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:30:37.926441 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.927701 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.932913 (+ 5212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.933975 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.934101 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.934262 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.937848 (+ 3586us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.937958 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1118,"thread_start_us":229,"threads_started":1}
I20260806 12:30:37.941416 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.922983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42377 (local address 127.0.0.1:39852)
0806 12:30:37.935319 (+ 12336us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:37.935336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:37.935518 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:37.936602 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:37.936606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:37.936628 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:30:37.936748 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.936754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.938374 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.938389 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.939678 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:37.939702 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.940968 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.940972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:37.941058 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.941092 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:30:37.941189 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 12:30:37.941256 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12258,"thread_start_us":123,"threads_started":1}
I20260806 12:30:37.942209 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.923525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.6.152.129:42377)
0806 12:30:37.935848 (+ 12323us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.935855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.935879 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:30:37.935961 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.935967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.936195 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.936400 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:30:37.936969 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.938240 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.939852 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.940638 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.940748 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.940899 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.941956 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.942058 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12156,"thread_start_us":118,"threads_started":1}
I20260806 12:30:37.941918 19009 tablet_service.cc:1467] Processing CreateTablet for tablet a5f31012a3a14816a40392493b148217 (DEFAULT_TABLE table=test-table [id=fd1d956df0454fa498570025500707a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:30:37.943274 19009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f31012a3a14816a40392493b148217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:30:37.945281 18728 tablet_service.cc:1467] Processing CreateTablet for tablet a5f31012a3a14816a40392493b148217 (DEFAULT_TABLE table=test-table [id=fd1d956df0454fa498570025500707a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:30:37.946354 18728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f31012a3a14816a40392493b148217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:30:37.954000 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:37.920653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.6.152.130:45385)
0806 12:30:37.925218 (+ 4565us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:37.925248 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:37.925268 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:30:37.925346 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:37.925350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:37.925519 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:37.925724 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:30:37.929811 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.931152 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.934384 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:37.935446 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:37.935573 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:37.935748 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:37.953684 (+ 17936us) server_negotiation.cc:299] Negotiation successful
0806 12:30:37.953829 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4462,"thread_start_us":138,"threads_started":1}
I20260806 12:30:37.957652 18870 tablet_service.cc:1467] Processing CreateTablet for tablet a5f31012a3a14816a40392493b148217 (DEFAULT_TABLE table=test-table [id=fd1d956df0454fa498570025500707a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:30:37.958847 18870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f31012a3a14816a40392493b148217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:30:37.959504 19097 tablet_bootstrap.cc:492] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: Bootstrap starting.
I20260806 12:30:37.962495 19097 tablet_bootstrap.cc:654] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: Neither blocks nor log segments found. Creating new log.
I20260806 12:30:37.968519 19097 log.cc:824] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: Log is configured to *not* fsync() on all Append() calls
I20260806 12:30:37.969894 19096 tablet_bootstrap.cc:492] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Bootstrap starting.
I20260806 12:30:37.973456 19096 tablet_bootstrap.cc:654] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Neither blocks nor log segments found. Creating new log.
I20260806 12:30:37.974829 19096 log.cc:824] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:30:37.975422 19099 tablet_bootstrap.cc:492] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Bootstrap starting.
I20260806 12:30:37.977504 19097 tablet_bootstrap.cc:492] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: No bootstrap required, opened a new log
I20260806 12:30:37.977790 19097 ts_tablet_manager.cc:1397] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260806 12:30:37.978912 19099 tablet_bootstrap.cc:654] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Neither blocks nor log segments found. Creating new log.
I20260806 12:30:37.980715 19099 log.cc:824] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Log is configured to *not* fsync() on all Append() calls
I20260806 12:30:37.983657 19096 tablet_bootstrap.cc:492] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: No bootstrap required, opened a new log
I20260806 12:30:37.983958 19096 ts_tablet_manager.cc:1397] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.006s
I20260806 12:30:37.984272 19097 raft_consensus.cc:348] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:37.989420 19096 raft_consensus.cc:348] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:37.989821 19096 raft_consensus.cc:374] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:30:37.989948 19096 raft_consensus.cc:729] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0bff7487fe4256bca8c579d5f562c2, State: Initialized, Role: FOLLOWER
I20260806 12:30:37.990612 19096 consensus_queue.cc:260] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:37.992156 19096 ts_tablet_manager.cc:1428] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 12:30:37.992651 19077 heartbeater.cc:503] Master 127.6.152.190:43151 was elected leader, sending a full tablet report...
I20260806 12:30:37.992707 19097 raft_consensus.cc:374] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:30:37.992877 19097 raft_consensus.cc:729] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaec30a7c8e43559a862164477d63db, State: Initialized, Role: FOLLOWER
I20260806 12:30:37.993510 19097 consensus_queue.cc:260] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:37.994851 19097 ts_tablet_manager.cc:1428] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: Time spent starting tablet: real 0.017s user 0.000s sys 0.008s
I20260806 12:30:37.994922 19099 tablet_bootstrap.cc:492] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: No bootstrap required, opened a new log
I20260806 12:30:37.995216 19099 ts_tablet_manager.cc:1397] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260806 12:30:37.995296 18795 heartbeater.cc:503] Master 127.6.152.190:43151 was elected leader, sending a full tablet report...
I20260806 12:30:38.001194 19099 raft_consensus.cc:348] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:38.001654 19099 raft_consensus.cc:374] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:30:38.001775 19099 raft_consensus.cc:729] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b26ea55a5d54bb49ab77b1b8fef2416, State: Initialized, Role: FOLLOWER
I20260806 12:30:38.002542 19099 consensus_queue.cc:260] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:38.003885 19099 ts_tablet_manager.cc:1428] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260806 12:30:38.004349 18937 heartbeater.cc:503] Master 127.6.152.190:43151 was elected leader, sending a full tablet report...
I20260806 12:30:38.074405 19103 raft_consensus.cc:482] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:30:38.074709 19103 raft_consensus.cc:504] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:38.076453 19103 leader_election.cc:288] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b26ea55a5d54bb49ab77b1b8fef2416 (127.6.152.130:45385), 3aaec30a7c8e43559a862164477d63db (127.6.152.129:42377)
W20260806 12:30:38.081602 18796 tablet.cc:2369] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:30:38.090771 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:38.077107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:45385 (local address 127.6.152.131:59105)
0806 12:30:38.077566 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:38.077584 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:38.077718 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:38.083676 (+ 5958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:38.083679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:38.083703 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:30:38.083914 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:38.083927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.087985 (+ 4058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.087989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:38.089255 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:38.089268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.090307 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.090311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:38.090440 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:38.090466 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:30:38.090544 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:30:38.090626 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351}
I20260806 12:30:38.092123 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:38.077932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:59105 (local address 127.6.152.130:45385)
0806 12:30:38.082216 (+ 4284us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:38.082221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:38.082239 (+ 18us) server_negotiation.cc:407] Connection header received
0806 12:30:38.082296 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:38.082299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:38.082350 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:38.082435 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:30:38.086945 (+ 4510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.087729 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.089447 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.090058 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.090184 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:38.090242 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:38.091907 (+ 1665us) server_negotiation.cc:299] Negotiation successful
0806 12:30:38.091993 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4187}
I20260806 12:30:38.092808 18872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f31012a3a14816a40392493b148217" candidate_uuid: "ae0bff7487fe4256bca8c579d5f562c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" is_pre_election: true
I20260806 12:30:38.093150 18872 raft_consensus.cc:2454] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0bff7487fe4256bca8c579d5f562c2 in term 0.
I20260806 12:30:38.093900 18962 leader_election.cc:302] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b26ea55a5d54bb49ab77b1b8fef2416, ae0bff7487fe4256bca8c579d5f562c2; no voters:
I20260806 12:30:38.095139 19103 raft_consensus.cc:2781] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:30:38.095242 19103 raft_consensus.cc:482] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:30:38.095288 19103 raft_consensus.cc:3037] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:30:38.097086 19103 raft_consensus.cc:504] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:38.098126 18872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f31012a3a14816a40392493b148217" candidate_uuid: "ae0bff7487fe4256bca8c579d5f562c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416"
I20260806 12:30:38.098286 18872 raft_consensus.cc:3037] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:30:38.100016 18872 raft_consensus.cc:2454] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0bff7487fe4256bca8c579d5f562c2 in term 1.
I20260806 12:30:38.100440 18962 leader_election.cc:302] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b26ea55a5d54bb49ab77b1b8fef2416, ae0bff7487fe4256bca8c579d5f562c2; no voters:
I20260806 12:30:38.102288 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:38.082961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42377 (local address 127.6.152.131:35473)
0806 12:30:38.090955 (+ 7994us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:38.090970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:30:38.091055 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:30:38.092664 (+ 1609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:30:38.092668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:30:38.092690 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:30:38.092772 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:38.092779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.093869 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.093873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:38.094945 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:30:38.094956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.101806 (+ 6850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.101810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:30:38.101919 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:38.101953 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:30:38.102029 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:30:38.102105 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7938,"thread_start_us":205,"threads_started":1}
I20260806 12:30:38.102731 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:38.083496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:35473 (local address 127.6.152.129:42377)
0806 12:30:38.083612 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:38.083617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:38.091163 (+ 7546us) server_negotiation.cc:407] Connection header received
0806 12:30:38.091227 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:38.091230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:38.091283 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:38.091357 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:30:38.092938 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.093704 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.100907 (+ 7203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.101536 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.101645 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:38.101700 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:38.102527 (+ 827us) server_negotiation.cc:299] Negotiation successful
0806 12:30:38.102616 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:30:38.103778 18748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f31012a3a14816a40392493b148217" candidate_uuid: "ae0bff7487fe4256bca8c579d5f562c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaec30a7c8e43559a862164477d63db" is_pre_election: true
I20260806 12:30:38.103878 19103 leader_election.cc:288] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [CANDIDATE]: Term 1 election: Requested vote from peers 2b26ea55a5d54bb49ab77b1b8fef2416 (127.6.152.130:45385), 3aaec30a7c8e43559a862164477d63db (127.6.152.129:42377)
I20260806 12:30:38.103991 19103 raft_consensus.cc:2781] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:30:38.104060 19103 raft_consensus.cc:686] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 1 LEADER]: Becoming Leader. State: Replica: ae0bff7487fe4256bca8c579d5f562c2, State: Running, Role: LEADER
I20260806 12:30:38.104122 18748 raft_consensus.cc:2454] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0bff7487fe4256bca8c579d5f562c2 in term 0.
I20260806 12:30:38.104408 19103 consensus_queue.cc:237] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } }
I20260806 12:30:38.104545 18747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f31012a3a14816a40392493b148217" candidate_uuid: "ae0bff7487fe4256bca8c579d5f562c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaec30a7c8e43559a862164477d63db"
I20260806 12:30:38.104658 18747 raft_consensus.cc:3037] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:30:38.106199 18747 raft_consensus.cc:2454] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0bff7487fe4256bca8c579d5f562c2 in term 1.
W20260806 12:30:38.110649 19078 tablet.cc:2369] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:30:38.113122 18573 catalog_manager.cc:5495] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae0bff7487fe4256bca8c579d5f562c2 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "ae0bff7487fe4256bca8c579d5f562c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0bff7487fe4256bca8c579d5f562c2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 } health_report { overall_health: UNKNOWN } } }
Aug 06 12:30:38 dist-test-slave-1xrh krb5kdc[18071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019437, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:30:38.165063 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:38.159911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.6.152.131:39311)
0806 12:30:38.160030 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:30:38.160035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:30:38.160050 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:30:38.160187 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:30:38.160191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:30:38.160257 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:30:38.160375 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:30:38.160807 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.161609 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.162899 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:30:38.163267 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:30:38.163374 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:30:38.163782 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:30:38.164349 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:30:38.164464 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 12:30:38.164782 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 12:30:38.164844 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 12:30:38.173012 18747 raft_consensus.cc:1261] T a5f31012a3a14816a40392493b148217 P 3aaec30a7c8e43559a862164477d63db [term 1 FOLLOWER]: Refusing update from remote peer ae0bff7487fe4256bca8c579d5f562c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:30:38.174346 19107 consensus_queue.cc:1035] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaec30a7c8e43559a862164477d63db" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:30:38.181838 18872 raft_consensus.cc:1261] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 1 FOLLOWER]: Refusing update from remote peer ae0bff7487fe4256bca8c579d5f562c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:30:38.182562 19107 consensus_queue.cc:1035] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b26ea55a5d54bb49ab77b1b8fef2416" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:30:38.194309 19116 mvcc.cc:205] Tried to move back new op lower bound from 7315535618746056704 to 7315535618479374336. Current Snapshot: MvccSnapshot[applied={T|T < 7315535618746056704 or (T in {7315535618746056704})}]
I20260806 12:30:38.198101 19112 mvcc.cc:205] Tried to move back new op lower bound from 7315535618746056704 to 7315535618479374336. Current Snapshot: MvccSnapshot[applied={T|T < 7315535618746056704}]
I20260806 12:30:38.208428 18573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:30:38.208611 18573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:30:38.211513 18573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6420f7974ef648b28c7248038e1ddf77: Sending DeleteTablet for 3 replicas of tablet a5f31012a3a14816a40392493b148217
I20260806 12:30:38.212445 18728 tablet_service.cc:1514] Processing DeleteTablet for tablet a5f31012a3a14816a40392493b148217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:30:38 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:39852
I20260806 12:30:38.212491 19009 tablet_service.cc:1514] Processing DeleteTablet for tablet a5f31012a3a14816a40392493b148217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:30:38 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:36154
I20260806 12:30:38.213032 18870 tablet_service.cc:1514] Processing DeleteTablet for tablet a5f31012a3a14816a40392493b148217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:30:38 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:33994
I20260806 12:30:38.214124 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 18658
I20260806 12:30:38.214591 19124 tablet_replica.cc:330] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: stopping tablet replica
I20260806 12:30:38.214993 19124 raft_consensus.cc:2229] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:30:38.215283 19124 raft_consensus.cc:2258] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:30:38.216885 19124 ts_tablet_manager.cc:1905] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:30:38.219659 19124 ts_tablet_manager.cc:1918] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:30:38.219753 19124 log.cc:1196] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/a5f31012a3a14816a40392493b148217
I20260806 12:30:38.220052 19124 ts_tablet_manager.cc:1939] T a5f31012a3a14816a40392493b148217 P 2b26ea55a5d54bb49ab77b1b8fef2416: Deleting consensus metadata
I20260806 12:30:38.221460 18557 catalog_manager.cc:4841] TS 2b26ea55a5d54bb49ab77b1b8fef2416 (127.6.152.130:45385): tablet a5f31012a3a14816a40392493b148217 (table test-table [id=fd1d956df0454fa498570025500707a9]) successfully deleted
I20260806 12:30:38.223392 19122 tablet_replica.cc:330] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: stopping tablet replica
I20260806 12:30:38.223742 19122 raft_consensus.cc:2229] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:30:38.224206 19122 raft_consensus.cc:2258] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:30:38.225687 19122 ts_tablet_manager.cc:1905] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:30:38.228327 19122 ts_tablet_manager.cc:1918] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:30:38.228502 19122 log.cc:1196] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/a5f31012a3a14816a40392493b148217
I20260806 12:30:38.228904 19122 ts_tablet_manager.cc:1939] T a5f31012a3a14816a40392493b148217 P ae0bff7487fe4256bca8c579d5f562c2: Deleting consensus metadata
I20260806 12:30:38.230005 18558 catalog_manager.cc:4841] TS ae0bff7487fe4256bca8c579d5f562c2 (127.6.152.131:39311): tablet a5f31012a3a14816a40392493b148217 (table test-table [id=fd1d956df0454fa498570025500707a9]) successfully deleted
W20260806 12:30:38.236886 18559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:42377 (error 108)
I20260806 12:30:38.237782 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 18804
I20260806 12:30:38.239389 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:30:38.239041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42377 (local address 127.0.0.1:39864)
0806 12:30:38.239193 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:30:38.239281 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:42377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 12:30:38.239619 18559 catalog_manager.cc:4584] TS 3aaec30a7c8e43559a862164477d63db (127.6.152.129:42377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5f31012a3a14816a40392493b148217: Network error: Client connection negotiation failed: client connection to 127.6.152.129:42377: connect: Connection refused (error 111)
I20260806 12:30:38.246157 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 18942
I20260806 12:30:38.254526 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 18539
I20260806 12:30:38.263520 6754 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 12:30:38.607771 6754 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 12:30:38.607903 6754 mini_ranger.cc:66] Stopping Ranger...
I20260806 12:30:38.952590 6754 mini_ranger.cc:68] Stopped Ranger
2026-08-06 12:30:38.952 UTC [18102] LOG: received smart shutdown request
2026-08-06 12:30:38.957 UTC [18102] LOG: background worker "logical replication launcher" (PID 18111) exited with exit code 1
2026-08-06 12:30:38.957 UTC [18106] LOG: shutting down
2026-08-06 12:30:39.110 UTC [18102] LOG: database system is shut down
2026-08-06T12:30:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50012 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:30:39 dist-test-slave-1xrh krb5kdc[19134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:30:39 dist-test-slave-1xrh krb5kdc[19134](info): set up 2 sockets
Aug 06 12:30:39 dist-test-slave-1xrh krb5kdc[19134](info): commencing operation
krb5kdc: starting...
W20260806 12:30:40.387051 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:30:40 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:30:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/rangeradmin_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/rangeradmin_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/rangerlookup_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/rangerlookup_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
I20260806 12:30:40.559898 6754 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknpcosN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/postgres -l logfile start
2026-08-06 12:30:43.078 UTC [19164] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 12:30:43.078 UTC [19164] LOG: listening on IPv4 address "127.6.152.148", port 44299
2026-08-06 12:30:43.081 UTC [19164] LOG: listening on Unix socket "/tmp/.s.PGSQL.44299"
2026-08-06 12:30:43.106 UTC [19167] LOG: database system was shut down at 2026-08-06 12:30:41 UTC
2026-08-06 12:30:43.110 UTC [19164] LOG: database system is ready to accept connections
I20260806 12:30:44.082870 6754 mini_postgres.cc:96] Postgres bound to 44299
2026-08-06 12:30:44.089 UTC [19176] FATAL: database "slave" does not exist
127.6.152.148:44299 - accepting connections
I20260806 12:30:44.090374 6754 mini_ranger.cc:161] Starting Ranger...
I20260806 12:30:44.105677 6754 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 12:30:44.583524 6754 mini_ranger.cc:87] Created ranger Postgres database
I20260806 12:30:44.583621 6754 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-admin
2026-08-06 12:30:44,858 [I] DB FLAVOR :POSTGRES
2026-08-06 12:30:44,859 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:30:44,859 [I] Checking connection..
2026-08-06 12:30:44,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 12:30:45,087 [I] Checking connection passed.
2026-08-06 12:30:45,087 [I] --------- Verifying version history table ---------
2026-08-06 12:30:45,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:30:45,359 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:30:45,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:30:45,604 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 12:30:45,604 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 12:30:45,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 12:30:45.864 UTC [19242] WARNING: there is no transaction in progress
2026-08-06 12:30:45,871 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 12:30:45,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 12:30:46,106 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 12:30:46,106 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:30:46,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 12:30:46,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 12:30:46,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 12:30:46,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:30:47,066 [I] Table x_portal_user does not exist in database ranger
2026-08-06 12:30:47,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:30:47,310 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 12:30:47,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:30:47,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:30:47,802 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 12:30:47,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 12:30:48.730 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.747 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.764 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.779 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.795 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.812 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.874 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.881 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.895 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.905 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.915 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.928 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.937 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.943 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.949 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:48.955 UTC [19353] WARNING: there is no transaction in progress
2026-08-06 12:30:49,559 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 12:30:49,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 12:30:49,802 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 12:30:49,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 12:30:50,066 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 12:30:50,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 12:30:50,307 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 12:30:50,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 12:30:50,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 12:30:50,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 12:30:51,031 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 12:30:51,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 12:30:51,257 [I] DB_PATCHES have already been applied
I20260806 12:30:51.263432 6754 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknpcosN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 12:30:51.263494 6754 mini_ranger.cc:193] Using host: 127.6.152.148
I20260806 12:30:51.267303 6754 mini_ranger.cc:227] Ranger admin URL: http://127.6.152.148:50193
Aug 06, 2026 12:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:30:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:30:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 12:30:52.423394 6754 mini_ranger.cc:160] Time spent starting Ranger: real 8.333s user 0.001s sys 0.010s
Aug 06, 2026 12:31:12 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:31:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 12:31:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:31:12 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:31:13 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 12:31:13.796608 6754 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.152.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.152.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/rangerkms_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.152.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/rangerkms_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.152.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.152.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.152.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/HTTP_127.6.152.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 12:31:13 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019473, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 12:31:13.885917 6754 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 12:31:13.895301 6754 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 12:31:14.791996 6754 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 12:31:14.792094 6754 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms
I20260806 12:31:14.792119 6754 mini_ranger_kms.cc:227] Using postgres at 127.6.152.148:44299
2026-08-06 12:31:14,962 [I] DB FLAVOR :POSTGRES
2026-08-06 12:31:14,962 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 12:31:14,962 [I] Checking connection
2026-08-06 12:31:14,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 12:31:15,186 [I] connection success
2026-08-06 12:31:15,186 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 12:31:15,186 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 12:31:15,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 12:31:15,416 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 12:31:15,417 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 12:31:15,417 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 12:31:15,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.152.148:44299/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 12:31:15,693 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 12:31:16.152885 6754 mini_ranger_kms.cc:310] Created kms service
I20260806 12:31:16.371891 6754 mini_ranger_kms.cc:326] Created kudu user
I20260806 12:31:16.417156 6754 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 12:31:16 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019473, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:16.799796 6754 mini_ranger_kms.cc:384] Added ranger policy
I20260806 12:31:16.800040 6754 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknpcosN/build/debug/bin/postgresql.jar:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknpcosN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 12:31:16.800062 6754 mini_ranger_kms.cc:240] Using host: 127.6.152.148
I20260806 12:31:16.804369 6754 mini_ranger_kms.cc:276] Ranger KMS PID: 19538
I20260806 12:31:16.804450 6754 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.152.148:37925
Aug 06, 2026 12:31:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:31:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:31:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 12:31:19.123888 6754 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.238s user 0.001s sys 0.153s
I20260806 12:31:19.124019 6754 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 12:31:19.124109 6754 mini_ranger_kms.cc:401] 127.6.152.148:37925/kms/v1/keys
Aug 06 12:31:19 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786019479, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.152.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 12:31:19 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019479, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.152.148@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
Aug 06 12:31:19 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019479, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.152.148@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
Aug 06 12:31:22 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:31:22.107295 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:46333
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:46333
--ranger_config_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:22.261250 19590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:22.261557 19590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:22.261639 19590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:22.266084 19590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:22.266183 19590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:22.266222 19590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:22.266278 19590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:22.266302 19590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:22.272142 19590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.152.148:37925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:46333
--ranger_config_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:46333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:22.273192 19590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:22.274271 19590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:22.282760 19595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:22.282773 19598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:22.282980 19596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:22.283363 19590 server_base.cc:1034] running on GCE node
I20260806 12:31:22.283972 19590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:22.285107 19590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:22.286437 19590 hybrid_clock.cc:648] HybridClock initialized: now 1786019482286391 us; error 38 us; skew 500 ppm
Aug 06 12:31:22 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:22.289333 19590 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:22.290279 19590 webserver.cc:460] Webserver started at http://127.6.152.190:46165/ using document root <none> and password file <none>
I20260806 12:31:22.290793 19590 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:22.290884 19590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:22.291102 19590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:31:22 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:22.351733 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "0d47db764edd4b76aa7de2cd7b2b20d4"
format_stamp: "Formatted at 2026-08-06 12:31:22 on dist-test-slave-1xrh"
server_key: "6614c1a29b92b234e2c2cbfa1f892c65"
server_key_iv: "d4fce31b7379fac4b479a4d06a0d1221"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:22.352396 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "0d47db764edd4b76aa7de2cd7b2b20d4"
format_stamp: "Formatted at 2026-08-06 12:31:22 on dist-test-slave-1xrh"
server_key: "6614c1a29b92b234e2c2cbfa1f892c65"
server_key_iv: "d4fce31b7379fac4b479a4d06a0d1221"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:22.356505 19590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 12:31:22.374092 19605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:22.375682 19590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:31:22.375825 19590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "0d47db764edd4b76aa7de2cd7b2b20d4"
format_stamp: "Formatted at 2026-08-06 12:31:22 on dist-test-slave-1xrh"
server_key: "6614c1a29b92b234e2c2cbfa1f892c65"
server_key_iv: "d4fce31b7379fac4b479a4d06a0d1221"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:22.375967 19590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:22.385785 19590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:22.388659 19590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:22.388870 19590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:22.397264 19590 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:46333
I20260806 12:31:22.397270 19660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:46333 every 8 connection(s)
I20260806 12:31:22.398216 19590 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:22.401006 19661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:22.406082 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 19590
I20260806 12:31:22.406231 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:31:22.406559 6754 external_mini_cluster.cc:1428] Setting key 4c3eeb88b1b8981ec8e8e1d035a3064f
I20260806 12:31:22.407017 19661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Bootstrap starting.
I20260806 12:31:22.409502 19661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:22.410225 19661 log.cc:824] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:22.412928 19661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: No bootstrap required, opened a new log
Aug 06 12:31:22 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:22.417994 19661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:22.418313 19661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:22.418409 19661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d47db764edd4b76aa7de2cd7b2b20d4, State: Initialized, Role: FOLLOWER
I20260806 12:31:22.418948 19661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:22.419190 19661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:22.419283 19661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:22.419399 19661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:22.420601 19661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:22.421003 19661 leader_election.cc:302] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d47db764edd4b76aa7de2cd7b2b20d4; no voters:
I20260806 12:31:22.421411 19661 leader_election.cc:288] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:31:22.421434 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:22.408057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.6.152.190:46333)
0806 12:31:22.408578 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:22.408599 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:22.408646 (+ 47us) server_negotiation.cc:407] Connection header received
0806 12:31:22.409588 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:22.409612 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:22.410078 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:22.410599 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:22.411573 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:22.412515 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:22.413494 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:22.413947 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:22.414108 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:22.416616 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:22.416622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:22.416634 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:22.416658 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:22.418367 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:22.418897 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:22.418901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:22.418905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:22.418999 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:22.419321 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:22.419324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:22.419326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:22.419759 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:22.419969 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:22.420300 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 12:31:22.420587 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":166,"threads_started":1}
I20260806 12:31:22.421471 19666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:22.422788 19666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 LEADER]: Becoming Leader. State: Replica: 0d47db764edd4b76aa7de2cd7b2b20d4, State: Running, Role: LEADER
I20260806 12:31:22.422935 19661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:22.423152 19666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:22.424705 19666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } } }
I20260806 12:31:22.424880 19666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:22.425139 19666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d47db764edd4b76aa7de2cd7b2b20d4. Latest consensus state: current_term: 1 leader_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } } }
I20260806 12:31:22.425678 19666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:22.425482 19661 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 12:31:22.426105 19681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:22.432385 19681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:22.442063 19681 catalog_manager.cc:1347] Generated new cluster ID: 3059c38426134763a05b0f988531bfc6
I20260806 12:31:22.442123 19681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:22.501631 19681 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:31:22.503278 19681 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:22.513746 19681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Generated new TSK 0
I20260806 12:31:22.514650 19681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:31:23.195664 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:46333
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:23.320092 19707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:23.320542 19707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:23.320662 19707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:23.327350 19707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:23.327431 19707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:23.327536 19707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:31:23.333830 19707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.152.148:37925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:23.334892 19707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:23.336007 19707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:23.345989 19713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:23.345952 19712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:23.346393 19715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:23.346537 19707 server_base.cc:1034] running on GCE node
I20260806 12:31:23.346928 19707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:23.347663 19707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:23.348860 19707 hybrid_clock.cc:648] HybridClock initialized: now 1786019483348800 us; error 77 us; skew 500 ppm
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:23.351977 19707 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:31:23.353006 19707 webserver.cc:460] Webserver started at http://127.6.152.129:44095/ using document root <none> and password file <none>
I20260806 12:31:23.353741 19707 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:23.353914 19707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:23.354187 19707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:23.368932 19707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "694a7004ccf84871a3a21169502cd816"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b0473ef514f122ea12be4e4edacf3a1c"
server_key_iv: "7899d4a8ca05190befe07db8a09338a4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:23.369901 19707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "694a7004ccf84871a3a21169502cd816"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b0473ef514f122ea12be4e4edacf3a1c"
server_key_iv: "7899d4a8ca05190befe07db8a09338a4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:23.375167 19707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 12:31:23.395970 19724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:23.397766 19707 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260806 12:31:23.397902 19707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "694a7004ccf84871a3a21169502cd816"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b0473ef514f122ea12be4e4edacf3a1c"
server_key_iv: "7899d4a8ca05190befe07db8a09338a4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:23.398056 19707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:23.435563 19707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:23.439282 19707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:23.439488 19707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:23.440178 19707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:23.441197 19707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:23.441314 19707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:23.441437 19707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:23.441522 19707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:23.452181 19707 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:37067
I20260806 12:31:23.453333 19707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:31:23.455577 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 19707
I20260806 12:31:23.455717 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:31:23.456061 6754 external_mini_cluster.cc:1428] Setting key 9a6d14df3edb08c038946464f0e51036
I20260806 12:31:23.456563 19841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:37067 every 8 connection(s)
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:23.493569 19844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:23.458802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:46333 (local address 127.6.152.129:57839)
0806 12:31:23.469387 (+ 10585us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:23.469454 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:23.470734 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:23.471568 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:23.471578 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:23.471958 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:23.472608 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:23.472626 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.473842 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.473850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:23.475543 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:23.475554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.476355 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.476360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:23.476563 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:23.484154 (+ 7591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:23.484177 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:23.487672 (+ 3495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:23.490100 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:23.490107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:23.490124 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:23.490521 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:23.490866 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:23.490869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:23.490871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:23.491045 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:23.491580 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:23.491586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:23.491923 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 12:31:23.492114 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 12:31:23.492443 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10300,"thread_start_us":143,"threads_started":1}
I20260806 12:31:23.494320 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:23.459804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:57839 (local address 127.6.152.190:46333)
0806 12:31:23.460475 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:23.460484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:23.469572 (+ 9088us) server_negotiation.cc:407] Connection header received
0806 12:31:23.471005 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:23.471008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:23.471065 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:23.471250 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:23.472788 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.473686 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.475696 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.476044 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.476157 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:23.487886 (+ 11729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:23.487890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:23.487893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:23.487922 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:23.489954 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:23.490645 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:23.490649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:23.490651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:23.490744 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:23.491186 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:23.491189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:23.491191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:23.491420 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:23.491526 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:23.494019 (+ 2493us) server_negotiation.cc:299] Negotiation successful
0806 12:31:23.494163 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":111,"threads_started":1}
I20260806 12:31:23.496104 19842 heartbeater.cc:347] Connected to a master server at 127.6.152.190:46333
I20260806 12:31:23.496335 19842 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:23.496863 19842 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:23.498678 19624 ts_manager.cc:194] Registered new tserver with Master: 694a7004ccf84871a3a21169502cd816 (127.6.152.129:37067)
I20260806 12:31:23.500280 19624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:57839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:31:23.503986 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:46333
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 12:31:23.629921 19850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:23.630244 19850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:23.630336 19850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:23.634791 19850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:23.634876 19850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:23.634986 19850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:31:23.640395 19850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.152.148:37925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:23.641471 19850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:23.642493 19850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:23.652372 19857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:23.652292 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:23.652303 19856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:23.653673 19850 server_base.cc:1034] running on GCE node
I20260806 12:31:23.654052 19850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:23.654744 19850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:23.655973 19850 hybrid_clock.cc:648] HybridClock initialized: now 1786019483655940 us; error 42 us; skew 500 ppm
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:23.659431 19850 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:31:23.660319 19850 webserver.cc:460] Webserver started at http://127.6.152.130:36581/ using document root <none> and password file <none>
I20260806 12:31:23.660885 19850 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:23.660992 19850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:23.661229 19850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:23.679337 19850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "8d7c5497fc3c4471b49a0b4689496fba"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b1222b1eb920e7cc7097c92ac1a080bd"
server_key_iv: "38468cf95a6354ef31d51568e3929dd4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:23.680171 19850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "8d7c5497fc3c4471b49a0b4689496fba"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b1222b1eb920e7cc7097c92ac1a080bd"
server_key_iv: "38468cf95a6354ef31d51568e3929dd4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:23.688498 19850 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260806 12:31:23.707480 19866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:23.709213 19850 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260806 12:31:23.709393 19850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "8d7c5497fc3c4471b49a0b4689496fba"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b1222b1eb920e7cc7097c92ac1a080bd"
server_key_iv: "38468cf95a6354ef31d51568e3929dd4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:23.709589 19850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:23.737521 19850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:23.741627 19850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:23.741928 19850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:23.742710 19850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:23.747668 19850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:23.747798 19850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:23.747924 19850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:23.748008 19850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:23.770682 19850 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:32983
I20260806 12:31:23.772125 19850 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:31:23.773767 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 19850
I20260806 12:31:23.773913 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:31:23.774227 6754 external_mini_cluster.cc:1428] Setting key 9b080134930acde65abde300eb8aaa97
I20260806 12:31:23.774483 19985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:32983 every 8 connection(s)
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:23.802436 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:23.776620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:44113 (local address 127.6.152.190:46333)
0806 12:31:23.776779 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:23.776784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:23.781481 (+ 4697us) server_negotiation.cc:407] Connection header received
0806 12:31:23.782948 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:23.782952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:23.783031 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:23.783095 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:23.784999 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.785956 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.792974 (+ 7018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.793409 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.793516 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:23.797122 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:23.797126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:23.797130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:23.797163 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:23.799178 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:23.799864 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:23.799870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:23.799872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:23.799987 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:23.800454 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:23.800458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:23.800460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:23.800712 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:23.800809 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:23.802088 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0806 12:31:23.802246 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:31:23.804677 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:23.776812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:46333 (local address 127.6.152.130:44113)
0806 12:31:23.781291 (+ 4479us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:23.781360 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:23.782637 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:23.783591 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:23.783602 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:23.784033 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:23.784783 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:23.784807 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.786139 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.786144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:23.787722 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:23.787734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:23.793774 (+ 6040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:23.793779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:23.794034 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:23.794862 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:23.794898 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:23.796871 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:23.799336 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:23.799344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:23.799362 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:23.799731 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:23.800116 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:23.800119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:23.800121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:23.800322 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:23.800905 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:23.800912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:23.801256 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 12:31:23.803038 (+ 1782us) client_negotiation.cc:240] Negotiation successful
0806 12:31:23.803462 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4199,"thread_start_us":168,"threads_started":1}
I20260806 12:31:23.807049 19986 heartbeater.cc:347] Connected to a master server at 127.6.152.190:46333
I20260806 12:31:23.807341 19986 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:23.807999 19986 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:23.809329 19624 ts_manager.cc:194] Registered new tserver with Master: 8d7c5497fc3c4471b49a0b4689496fba (127.6.152.130:32983)
I20260806 12:31:23.810115 19624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:44113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:31:23.836568 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:46333
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:23.974241 19991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:23.974663 19991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:23.974794 19991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:23.979426 19991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:23.979503 19991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:23.979614 19991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:31:23.986662 19991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.152.148:37925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:23.987788 19991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:23.988862 19991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:23.998787 19997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:23.999004 19996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:23.999207 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:23.999352 19991 server_base.cc:1034] running on GCE node
I20260806 12:31:23.999790 19991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:24.000480 19991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:24.001654 19991 hybrid_clock.cc:648] HybridClock initialized: now 1786019484001628 us; error 41 us; skew 500 ppm
Aug 06 12:31:23 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:24.004585 19991 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:31:24.005376 19991 webserver.cc:460] Webserver started at http://127.6.152.131:32955/ using document root <none> and password file <none>
I20260806 12:31:24.006072 19991 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:24.006170 19991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:24.006407 19991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:24.023237 19991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "f7fc075232ef4ece8867a25aa80b47b2"
format_stamp: "Formatted at 2026-08-06 12:31:24 on dist-test-slave-1xrh"
server_key: "7079c075a21d57b1366e0d0100cffdf4"
server_key_iv: "b02fa0e49fd0b5da7f0e714ca1210b2d"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:24.023941 19991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "f7fc075232ef4ece8867a25aa80b47b2"
format_stamp: "Formatted at 2026-08-06 12:31:24 on dist-test-slave-1xrh"
server_key: "7079c075a21d57b1366e0d0100cffdf4"
server_key_iv: "b02fa0e49fd0b5da7f0e714ca1210b2d"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:24.028689 19991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:24.066381 20006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.068163 19991 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.000s
I20260806 12:31:24.068313 19991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "f7fc075232ef4ece8867a25aa80b47b2"
format_stamp: "Formatted at 2026-08-06 12:31:24 on dist-test-slave-1xrh"
server_key: "7079c075a21d57b1366e0d0100cffdf4"
server_key_iv: "b02fa0e49fd0b5da7f0e714ca1210b2d"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:24.068449 19991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:24.085129 19991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:24.088474 19991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:24.088655 19991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:24.089264 19991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:24.090234 19991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:24.090291 19991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.090350 19991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:24.090377 19991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.104291 19991 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:39667
I20260806 12:31:24.104420 20121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:39667 every 8 connection(s)
I20260806 12:31:24.105492 19991 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:31:24.110061 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 19991
I20260806 12:31:24.110203 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:31:24.110538 6754 external_mini_cluster.cc:1428] Setting key 5a53ea5f88377d9b1c44272b2ae5d7de
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:24.122638 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:24.109847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:45611 (local address 127.6.152.190:46333)
0806 12:31:24.109976 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:24.109981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:24.110000 (+ 19us) server_negotiation.cc:407] Connection header received
0806 12:31:24.110713 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:24.110717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:24.110771 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:24.110821 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:24.112477 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.113260 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.114994 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.115413 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.115529 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:24.118580 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:24.118584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:24.118587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:24.118615 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:24.120323 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:24.120932 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:24.120936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:24.120938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:24.121025 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:24.121431 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:24.121434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:24.121436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:24.121643 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:24.121732 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:24.122363 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 12:31:24.122485 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:31:24.124286 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:24.108720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:46333 (local address 127.6.152.131:45611)
0806 12:31:24.109343 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:24.109402 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:24.110470 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:24.111240 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:24.111249 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:24.111623 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:24.112315 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:24.112333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.113421 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.113425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:24.114842 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:24.114853 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.115573 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.115577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:24.115778 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:24.116532 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:24.116552 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:24.118385 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:24.120452 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:24.120458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:24.120473 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:24.120824 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:24.121127 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:24.121130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:24.121132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:24.121317 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:24.121761 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:24.121767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:24.122082 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 12:31:24.122304 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 12:31:24.122628 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":181,"threads_started":1}
I20260806 12:31:24.125506 20122 heartbeater.cc:347] Connected to a master server at 127.6.152.190:46333
I20260806 12:31:24.125751 20122 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.126271 20122 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:24.127451 19624 ts_manager.cc:194] Registered new tserver with Master: f7fc075232ef4ece8867a25aa80b47b2 (127.6.152.131:39667)
I20260806 12:31:24.128381 19624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:45611
I20260806 12:31:24.135938 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:31:24.136817 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 19707
I20260806 12:31:24.149314 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 19850
I20260806 12:31:24.157090 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 19991
I20260806 12:31:24.165185 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 19590
I20260806 12:31:24.173727 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:46333
--webserver_interface=127.6.152.190
--webserver_port=46165
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:46333
--ranger_config_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:24.281687 20125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:24.281942 20125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:24.282012 20125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:24.286006 20125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:24.286058 20125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:24.286077 20125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:24.286092 20125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:24.286108 20125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:24.291224 20125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:46333
--ranger_config_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:46333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:24.292187 20125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:24.293182 20125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:24.301833 20133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:24.301885 20125 server_base.cc:1034] running on GCE node
W20260806 12:31:24.301859 20130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:24.302050 20131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:24.302728 20125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:24.303908 20125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:24.305075 20125 hybrid_clock.cc:648] HybridClock initialized: now 1786019484305042 us; error 47 us; skew 500 ppm
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:24.307989 20125 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:24.308892 20125 webserver.cc:460] Webserver started at http://127.6.152.190:46165/ using document root <none> and password file <none>
I20260806 12:31:24.309439 20125 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:24.309561 20125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:24.359544 20125 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 12:31:24.361867 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.363099 20125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:24.363301 20125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "0d47db764edd4b76aa7de2cd7b2b20d4"
format_stamp: "Formatted at 2026-08-06 12:31:22 on dist-test-slave-1xrh"
server_key: "6614c1a29b92b234e2c2cbfa1f892c65"
server_key_iv: "d4fce31b7379fac4b479a4d06a0d1221"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:24.363870 20125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:24.396023 20125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:24.398699 20125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:24.399187 20125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:24.406057 20125 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:46333
I20260806 12:31:24.406066 20195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:46333 every 8 connection(s)
I20260806 12:31:24.407162 20125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:24.411454 20196 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:31:24.411799 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20125
I20260806 12:31:24.412341 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:37067
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:46333
--webserver_port=44095
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:24.413784 20196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Bootstrap starting.
I20260806 12:31:24.430553 20196 log.cc:824] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:24.437083 20196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:31:24.437479 20196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Bootstrap complete.
I20260806 12:31:24.441635 20196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:24.442013 20196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d47db764edd4b76aa7de2cd7b2b20d4, State: Initialized, Role: FOLLOWER
I20260806 12:31:24.442489 20196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:24.442610 20196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:24.442698 20196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:24.442795 20196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:31:24.444743 20196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:24.445086 20196 leader_election.cc:302] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d47db764edd4b76aa7de2cd7b2b20d4; no voters:
I20260806 12:31:24.445417 20196 leader_election.cc:288] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:31:24.445519 20200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:31:24.445724 20200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [term 2 LEADER]: Becoming Leader. State: Replica: 0d47db764edd4b76aa7de2cd7b2b20d4, State: Running, Role: LEADER
I20260806 12:31:24.446018 20200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } }
I20260806 12:31:24.446688 20196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:24.447525 20201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } } }
I20260806 12:31:24.447647 20201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:24.447582 20202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d47db764edd4b76aa7de2cd7b2b20d4. Latest consensus state: current_term: 2 leader_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d47db764edd4b76aa7de2cd7b2b20d4" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46333 } } }
I20260806 12:31:24.447698 20202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:24.449124 20196 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 12:31:24.449227 20207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:24.455060 20207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:24.455943 20207 catalog_manager.cc:1259] Loaded cluster ID: 3059c38426134763a05b0f988531bfc6
I20260806 12:31:24.456012 20207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:24.459676 20207 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:24.460554 20207 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Loaded TSK: 0
I20260806 12:31:24.461760 20207 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:31:24.558144 20198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:24.558472 20198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:24.558537 20198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:24.564599 20198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:24.564673 20198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:24.564759 20198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:31:24.572682 20198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:37067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=44095
--tserver_master_addrs=127.6.152.190:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:24.573949 20198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:24.575305 20198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:24.587558 20234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:24.588114 20233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:24.588167 20236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:24.589243 20198 server_base.cc:1034] running on GCE node
I20260806 12:31:24.589694 20198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:24.590407 20198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:24.593695 20198 hybrid_clock.cc:648] HybridClock initialized: now 1786019484593603 us; error 97 us; skew 500 ppm
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:24.599623 20198 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:31:24.600710 20198 webserver.cc:460] Webserver started at http://127.6.152.129:44095/ using document root <none> and password file <none>
I20260806 12:31:24.601424 20198 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:24.601516 20198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:24.628119 20198 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 12:31:24.633302 20245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.634807 20198 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 12:31:24.634948 20198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "694a7004ccf84871a3a21169502cd816"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b0473ef514f122ea12be4e4edacf3a1c"
server_key_iv: "7899d4a8ca05190befe07db8a09338a4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:24.635581 20198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:24.672170 20198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:24.675766 20198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:24.675982 20198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:24.676615 20198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:24.677651 20198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:24.677722 20198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.677788 20198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:24.677825 20198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.689415 20198 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:37067
I20260806 12:31:24.689505 20359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:37067 every 8 connection(s)
I20260806 12:31:24.690623 20198 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:31:24.693707 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20198
I20260806 12:31:24.694304 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:32983
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:46333
--webserver_port=36581
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:24.712827 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:24.693996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:37007 (local address 127.6.152.190:46333)
0806 12:31:24.694537 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:24.694560 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:24.695013 (+ 453us) server_negotiation.cc:407] Connection header received
0806 12:31:24.696423 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:24.696444 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:24.696916 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:24.697244 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:24.699030 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.700618 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.702307 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.703036 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.703199 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:24.706719 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:24.706727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:24.706745 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:24.706776 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:24.708924 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:24.709612 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:24.709620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:24.709628 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:24.709765 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:24.710182 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:24.710186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:24.710188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:24.710642 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:24.710751 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:24.711490 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 12:31:24.711753 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":130,"threads_started":1}
I20260806 12:31:24.714715 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:24.694094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:46333 (local address 127.6.152.129:37007)
0806 12:31:24.694824 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:24.694875 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:24.695859 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:24.697851 (+ 1992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:24.697860 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:24.698210 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:24.698833 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:24.698852 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.700790 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.700794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:24.702151 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:24.702162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.703463 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.703467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:24.703645 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:24.704315 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:24.704337 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:24.706485 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:24.709071 (+ 2586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:24.709077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:24.709090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:24.709471 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:24.709898 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:24.709901 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:24.709904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:24.710068 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:24.710847 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:24.710853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:24.711157 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 12:31:24.713568 (+ 2411us) client_negotiation.cc:240] Negotiation successful
0806 12:31:24.713882 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":149,"threads_started":1}
I20260806 12:31:24.717926 20360 heartbeater.cc:347] Connected to a master server at 127.6.152.190:46333
I20260806 12:31:24.718153 20360 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.718690 20360 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
W20260806 12:31:24.721230 20360 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:46333 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 12:31:24.721356 20360 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.721529 20360 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:24.722457 20360 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.722623 20360 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
W20260806 12:31:24.723448 20360 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:31:24.836485 20364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:24.836704 20364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:24.836743 20364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:24.840961 20364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:24.841035 20364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:24.841130 20364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:31:24.846534 20364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:32983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=36581
--tserver_master_addrs=127.6.152.190:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:24.847700 20364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:24.848943 20364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:24.858525 20369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:24.858541 20370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:24.858820 20372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:24.859424 20364 server_base.cc:1034] running on GCE node
I20260806 12:31:24.859872 20364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:24.860620 20364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:24.868026 20364 hybrid_clock.cc:648] HybridClock initialized: now 1786019484867995 us; error 50 us; skew 500 ppm
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:24.871718 20364 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:31:24.872835 20364 webserver.cc:460] Webserver started at http://127.6.152.130:36581/ using document root <none> and password file <none>
I20260806 12:31:24.873561 20364 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:24.873665 20364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:24.910768 20364 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.000s
I20260806 12:31:24.913371 20380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.914701 20364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:31:24.914842 20364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "8d7c5497fc3c4471b49a0b4689496fba"
format_stamp: "Formatted at 2026-08-06 12:31:23 on dist-test-slave-1xrh"
server_key: "b1222b1eb920e7cc7097c92ac1a080bd"
server_key_iv: "38468cf95a6354ef31d51568e3929dd4"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:24.915527 20364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:24.953866 20364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:24.957578 20364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:24.957796 20364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:24.958537 20364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:24.959554 20364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:24.959614 20364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.959666 20364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:24.959695 20364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:24.971129 20364 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:32983
I20260806 12:31:24.971184 20499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:32983 every 8 connection(s)
I20260806 12:31:24.973255 20364 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:31:24.974516 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20364
I20260806 12:31:24.975035 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.152.148:37925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:39667
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:46333
--webserver_port=32955
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:31:24 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:24.992585 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:24.980482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:56489 (local address 127.6.152.190:46333)
0806 12:31:24.980622 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:24.980626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:24.981549 (+ 923us) server_negotiation.cc:407] Connection header received
0806 12:31:24.982805 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:24.982809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:24.982891 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:24.982954 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:24.984306 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.984839 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.985960 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.986237 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.986354 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:24.988855 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:24.988859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:24.988862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:24.988894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:24.990321 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:24.990998 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:24.991002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:24.991004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:24.991101 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:24.991504 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:24.991508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:24.991509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:24.991682 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:24.991787 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:24.992272 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 12:31:24.992409 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:31:24.993728 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:24.980936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:46333 (local address 127.6.152.130:56489)
0806 12:31:24.981391 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:24.981443 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:24.982539 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:24.983389 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:24.983397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:24.983664 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:24.984146 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:24.984159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.984968 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.984972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:24.985814 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:24.985833 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:24.986387 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:24.986391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:24.986564 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:24.987346 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:24.987363 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:24.988696 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:24.990475 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:24.990481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:24.990495 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:24.990874 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:24.991253 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:24.991255 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:24.991257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:24.991402 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:24.991800 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:24.991808 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:24.992045 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 12:31:24.992230 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 12:31:24.992518 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":129,"threads_started":1}
I20260806 12:31:24.994879 20500 heartbeater.cc:347] Connected to a master server at 127.6.152.190:46333
I20260806 12:31:24.995102 20500 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.995592 20500 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
W20260806 12:31:24.997081 20500 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:46333 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 12:31:24.997207 20500 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.997385 20500 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:24.998138 20500 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:24.998317 20500 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
W20260806 12:31:24.999045 20500 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:31:25.130270 20502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:25.130542 20502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:25.130620 20502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:25.135854 20502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:25.135941 20502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:25.136026 20502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:31:25.142736 20502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.152.148:37925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:39667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=32955
--tserver_master_addrs=127.6.152.190:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:25.143922 20502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:25.145103 20502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:25.157397 20508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:25.157416 20511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:25.157614 20502 server_base.cc:1034] running on GCE node
W20260806 12:31:25.157655 20509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:25.158427 20502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:25.159188 20502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:25.160386 20502 hybrid_clock.cc:648] HybridClock initialized: now 1786019485160363 us; error 35 us; skew 500 ppm
Aug 06 12:31:25 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:25.163563 20502 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:31:25.164678 20502 webserver.cc:460] Webserver started at http://127.6.152.131:32955/ using document root <none> and password file <none>
I20260806 12:31:25.165228 20502 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:25.165337 20502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 12:31:25 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:25.197499 20502 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.002s sys 0.000s
I20260806 12:31:25.200718 20519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:25.202375 20502 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 12:31:25.202512 20502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "f7fc075232ef4ece8867a25aa80b47b2"
format_stamp: "Formatted at 2026-08-06 12:31:24 on dist-test-slave-1xrh"
server_key: "7079c075a21d57b1366e0d0100cffdf4"
server_key_iv: "b02fa0e49fd0b5da7f0e714ca1210b2d"
server_key_version: "kuduclusterkey@0"
I20260806 12:31:25.203325 20502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:25.225148 20502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:25.228863 20502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:25.229097 20502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:25.229821 20502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:25.230849 20502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:25.230917 20502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:25.230975 20502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:25.231011 20502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:25.243031 20502 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:39667
I20260806 12:31:25.243062 20646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:39667 every 8 connection(s)
I20260806 12:31:25.244266 20502 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:31:25.246672 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20502
Aug 06 12:31:25 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:25.261780 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:25.246223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:42835 (local address 127.6.152.190:46333)
0806 12:31:25.246383 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:25.246388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:25.247603 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 12:31:25.249253 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:25.249257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:25.249321 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:25.249421 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:25.250841 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:25.251719 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:25.253401 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:25.253662 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:25.253751 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:25.257377 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:25.257381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:25.257384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:25.257418 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:25.259314 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:25.259917 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:25.259922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:25.259925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:25.260013 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:25.260474 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:25.260478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:25.260480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:25.260726 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:25.260844 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:25.261485 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 12:31:25.261611 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:31:25.263320 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:25.246727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:46333 (local address 127.6.152.131:42835)
0806 12:31:25.247454 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:25.247508 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:25.248960 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:25.249620 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:25.249633 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:25.249994 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:25.250647 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:25.250670 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:25.251876 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:25.251880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:25.253262 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:25.253275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:25.253873 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:25.253878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:25.254145 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:25.254854 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:25.254874 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:25.257174 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:25.259442 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:25.259448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:25.259462 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:25.259786 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:25.260134 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:25.260137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:25.260139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:25.260345 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:25.260905 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:25.260912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:25.261206 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 12:31:25.261961 (+ 755us) client_negotiation.cc:240] Negotiation successful
0806 12:31:25.262287 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":128,"threads_started":1}
I20260806 12:31:25.263828 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:25.247862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.6.152.190:46333)
0806 12:31:25.248212 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:25.248221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:25.254776 (+ 6555us) server_negotiation.cc:407] Connection header received
0806 12:31:25.254814 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:25.254816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:25.254854 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:25.254960 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:25.255930 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:25.256721 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:25.257647 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:25.258029 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:25.258150 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:25.259640 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:25.259644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:25.259646 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:25.259670 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:25.261486 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:25.262179 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:25.262181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:25.262183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:25.262258 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:25.262559 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:25.262569 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:25.262572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:25.262728 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:25.262944 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:25.263099 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 12:31:25.263243 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"spinlock_wait_cycles":3968,"thread_start_us":118,"threads_started":1}
I20260806 12:31:25.264617 20648 heartbeater.cc:347] Connected to a master server at 127.6.152.190:46333
I20260806 12:31:25.264880 20648 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:25.265466 20648 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:25.266966 20158 ts_manager.cc:194] Registered new tserver with Master: f7fc075232ef4ece8867a25aa80b47b2 (127.6.152.131:39667)
I20260806 12:31:25.268098 20158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:42835
Aug 06 12:31:25 dist-test-slave-1xrh krb5kdc[19134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786019485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:25.723691 20360 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:25.724022 20360 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:25.724900 20158 ts_manager.cc:194] Registered new tserver with Master: 694a7004ccf84871a3a21169502cd816 (127.6.152.129:37067)
I20260806 12:31:25.725718 20158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:37007
Aug 06 12:31:25 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:25.999245 20500 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:25.999550 20500 heartbeater.cc:511] Master 127.6.152.190:46333 requested a full tablet report, sending...
I20260806 12:31:26.002655 20158 ts_manager.cc:194] Registered new tserver with Master: 8d7c5497fc3c4471b49a0b4689496fba (127.6.152.130:32983)
I20260806 12:31:26.003477 20158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:56489
I20260806 12:31:26.013185 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:31:26.024384 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.014900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.6.152.190:46333)
0806 12:31:26.015570 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.015574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.015930 (+ 356us) server_negotiation.cc:407] Connection header received
0806 12:31:26.015972 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.015975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.016024 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.016099 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:26.017023 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.017873 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.019093 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.019386 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.019511 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.020955 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:26.020959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:26.020961 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:26.020990 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:26.022577 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:26.023002 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:26.023006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:26.023008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:26.023092 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:26.023419 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:26.023422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:26.023424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:26.023598 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:26.023739 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.023984 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.024112 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":93,"threads_started":1}
I20260806 12:31:26.027998 20158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:31:26.031428 20158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:31:26.049042 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.041669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:32983 (local address 127.0.0.1:48592)
0806 12:31:26.043621 (+ 1952us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.043636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:26.044178 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:26.044650 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:26.044656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:26.044679 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:26.044812 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.044819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.046126 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.046130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.047075 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.047084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.048486 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.048490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.048608 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.048645 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 12:31:26.048780 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 12:31:26.048897 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1898,"thread_start_us":88,"threads_started":1}
I20260806 12:31:26.049620 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.042211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.6.152.130:32983)
0806 12:31:26.043483 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.043492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.044127 (+ 635us) server_negotiation.cc:407] Connection header received
0806 12:31:26.044342 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.044346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.044509 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.044662 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:26.044943 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.045998 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.047238 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.048173 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.048273 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.048423 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.049392 (+ 969us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.049493 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":197,"threads_started":1}
I20260806 12:31:26.052700 20413 tablet_service.cc:1467] Processing CreateTablet for tablet 027ca27c857c49718f311c810caacb08 (DEFAULT_TABLE table=test-table [id=8f1bf5c6173d465eb26245df5d713b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:26.053820 20413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 027ca27c857c49718f311c810caacb08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:26.054587 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.043436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:39667 (local address 127.0.0.1:49522)
0806 12:31:26.049819 (+ 6383us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.049833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:26.049906 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:26.050560 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:26.050563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:26.050578 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:26.050647 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.050652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.052017 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.052020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.053042 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.053053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.054226 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.054229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.054309 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.054343 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:31:26.054410 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 12:31:26.054469 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6333,"thread_start_us":100,"threads_started":1}
I20260806 12:31:26.055001 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.043851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49522 (local address 127.6.152.131:39667)
0806 12:31:26.044115 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.044122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.050004 (+ 5882us) server_negotiation.cc:407] Connection header received
0806 12:31:26.050086 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.050091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.050275 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.050419 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:26.050768 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.051887 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.053179 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.053943 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.054041 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.054167 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.054784 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.054882 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":102,"threads_started":1}
I20260806 12:31:26.057581 20570 tablet_service.cc:1467] Processing CreateTablet for tablet 027ca27c857c49718f311c810caacb08 (DEFAULT_TABLE table=test-table [id=8f1bf5c6173d465eb26245df5d713b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:26.058560 20570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 027ca27c857c49718f311c810caacb08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:26.060082 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.041408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37067 (local address 127.0.0.1:41918)
0806 12:31:26.042370 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.042430 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:26.042556 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:26.055820 (+ 13264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:26.055824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:26.055842 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:26.055935 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.055942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.057543 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.057546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.058664 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.058675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.059722 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.059726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.059820 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.059845 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 12:31:26.059897 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 12:31:26.059960 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":102,"threads_started":1}
I20260806 12:31:26.060822 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.041509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.6.152.129:37067)
0806 12:31:26.055365 (+ 13856us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.055372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.055397 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:31:26.055471 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.055476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.055661 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.056092 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:26.056268 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.057295 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.058824 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.059578 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.060358 (+ 780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.060504 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.060554 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.060638 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13727,"thread_start_us":109,"threads_started":1}
I20260806 12:31:26.063416 20294 tablet_service.cc:1467] Processing CreateTablet for tablet 027ca27c857c49718f311c810caacb08 (DEFAULT_TABLE table=test-table [id=8f1bf5c6173d465eb26245df5d713b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:26.064390 20294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 027ca27c857c49718f311c810caacb08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:26.071487 20675 tablet_bootstrap.cc:492] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Bootstrap starting.
I20260806 12:31:26.073755 20675 tablet_bootstrap.cc:654] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:26.074574 20675 log.cc:824] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:26.075287 20676 tablet_bootstrap.cc:492] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Bootstrap starting.
I20260806 12:31:26.077345 20676 tablet_bootstrap.cc:654] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:26.078106 20676 log.cc:824] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:26.079834 20675 tablet_bootstrap.cc:492] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: No bootstrap required, opened a new log
I20260806 12:31:26.080065 20675 ts_tablet_manager.cc:1397] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20260806 12:31:26.080194 20676 tablet_bootstrap.cc:492] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: No bootstrap required, opened a new log
I20260806 12:31:26.080417 20676 ts_tablet_manager.cc:1397] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 12:31:26.085253 20676 raft_consensus.cc:348] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.085614 20676 raft_consensus.cc:374] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:26.085723 20676 raft_consensus.cc:729] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7c5497fc3c4471b49a0b4689496fba, State: Initialized, Role: FOLLOWER
I20260806 12:31:26.085775 20675 raft_consensus.cc:348] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.086062 20675 raft_consensus.cc:374] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:26.086126 20675 raft_consensus.cc:729] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7fc075232ef4ece8867a25aa80b47b2, State: Initialized, Role: FOLLOWER
I20260806 12:31:26.086409 20676 consensus_queue.cc:260] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.086584 20675 consensus_queue.cc:260] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.087608 20676 ts_tablet_manager.cc:1428] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 12:31:26.087728 20675 ts_tablet_manager.cc:1428] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20260806 12:31:26.088059 20500 heartbeater.cc:503] Master 127.6.152.190:46333 was elected leader, sending a full tablet report...
I20260806 12:31:26.088321 20677 tablet_bootstrap.cc:492] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: Bootstrap starting.
I20260806 12:31:26.091077 20677 tablet_bootstrap.cc:654] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:26.092044 20677 log.cc:824] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:26.092624 20648 heartbeater.cc:503] Master 127.6.152.190:46333 was elected leader, sending a full tablet report...
I20260806 12:31:26.095983 20677 tablet_bootstrap.cc:492] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: No bootstrap required, opened a new log
I20260806 12:31:26.096194 20677 ts_tablet_manager.cc:1397] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 12:31:26.099886 20677 raft_consensus.cc:348] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.100160 20677 raft_consensus.cc:374] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:26.100215 20677 raft_consensus.cc:729] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 694a7004ccf84871a3a21169502cd816, State: Initialized, Role: FOLLOWER
I20260806 12:31:26.100934 20677 consensus_queue.cc:260] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.101621 20360 heartbeater.cc:503] Master 127.6.152.190:46333 was elected leader, sending a full tablet report...
I20260806 12:31:26.102051 20677 ts_tablet_manager.cc:1428] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 12:31:26.193215 20361 tablet.cc:2369] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:31:26.228598 20501 tablet.cc:2369] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:31:26.245952 20649 tablet.cc:2369] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 12:31:26 dist-test-slave-1xrh krb5kdc[19134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for HTTP/127.6.152.148@KRBTEST.COM
I20260806 12:31:26.431320 20681 raft_consensus.cc:482] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:26.431607 20681 raft_consensus.cc:504] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.433200 20681 leader_election.cc:288] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 694a7004ccf84871a3a21169502cd816 (127.6.152.129:37067), 8d7c5497fc3c4471b49a0b4689496fba (127.6.152.130:32983)
I20260806 12:31:26.441164 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.434358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58501 (local address 127.6.152.129:37067)
0806 12:31:26.434567 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.434573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.436375 (+ 1802us) server_negotiation.cc:407] Connection header received
0806 12:31:26.436662 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.436666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.436723 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.436781 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:26.437271 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.438187 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.439881 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.440400 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.440500 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.440549 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.440925 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.441010 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 12:31:26.441720 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.433727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37067 (local address 127.6.152.131:58501)
0806 12:31:26.436237 (+ 2510us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.436253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:26.436491 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:26.436941 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:26.436944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:26.436971 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:26.437110 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.437116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.438422 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.438426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.439730 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.439741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.440642 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.440646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.440755 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.440780 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 12:31:26.441463 (+ 683us) client_negotiation.cc:240] Negotiation successful
0806 12:31:26.441541 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2413,"thread_start_us":171,"threads_started":1}
I20260806 12:31:26.442433 20296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027ca27c857c49718f311c810caacb08" candidate_uuid: "f7fc075232ef4ece8867a25aa80b47b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "694a7004ccf84871a3a21169502cd816" is_pre_election: true
I20260806 12:31:26.442765 20296 raft_consensus.cc:2454] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7fc075232ef4ece8867a25aa80b47b2 in term 0.
I20260806 12:31:26.443378 20522 leader_election.cc:302] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 694a7004ccf84871a3a21169502cd816, f7fc075232ef4ece8867a25aa80b47b2; no voters:
I20260806 12:31:26.443745 20681 raft_consensus.cc:2781] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:31:26.443836 20681 raft_consensus.cc:482] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:31:26.443893 20681 raft_consensus.cc:3037] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:26.445571 20681 raft_consensus.cc:504] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.445996 20681 leader_election.cc:288] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [CANDIDATE]: Term 1 election: Requested vote from peers 694a7004ccf84871a3a21169502cd816 (127.6.152.129:37067), 8d7c5497fc3c4471b49a0b4689496fba (127.6.152.130:32983)
I20260806 12:31:26.446992 20296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027ca27c857c49718f311c810caacb08" candidate_uuid: "f7fc075232ef4ece8867a25aa80b47b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "694a7004ccf84871a3a21169502cd816"
I20260806 12:31:26.447136 20296 raft_consensus.cc:3037] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:26.448417 20296 raft_consensus.cc:2454] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7fc075232ef4ece8867a25aa80b47b2 in term 1.
I20260806 12:31:26.448726 20522 leader_election.cc:302] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 694a7004ccf84871a3a21169502cd816, f7fc075232ef4ece8867a25aa80b47b2; no voters:
I20260806 12:31:26.448894 20681 raft_consensus.cc:2781] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:26.449110 20681 raft_consensus.cc:686] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 1 LEADER]: Becoming Leader. State: Replica: f7fc075232ef4ece8867a25aa80b47b2, State: Running, Role: LEADER
I20260806 12:31:26.449452 20681 consensus_queue.cc:237] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } }
I20260806 12:31:26.453107 20159 catalog_manager.cc:5495] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7fc075232ef4ece8867a25aa80b47b2 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "f7fc075232ef4ece8867a25aa80b47b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7fc075232ef4ece8867a25aa80b47b2" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 39667 } health_report { overall_health: HEALTHY } } }
I20260806 12:31:26.462965 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.435202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:58777 (local address 127.6.152.130:32983)
0806 12:31:26.446972 (+ 11770us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.446976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.446991 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:31:26.447046 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.447048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.447119 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.447270 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:26.447556 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.448271 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.461550 (+ 13279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.462201 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.462325 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.462384 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.462735 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.462830 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11649}
I20260806 12:31:26.463984 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.434730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:32983 (local address 127.6.152.131:58777)
0806 12:31:26.442221 (+ 7491us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.442245 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:26.442317 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:26.447279 (+ 4962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:26.447282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:26.447298 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:26.447377 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.447384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.457279 (+ 9895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.457282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.460903 (+ 3621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:26.460916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.462485 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.462488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:26.462577 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.462603 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:31:26.463720 (+ 1117us) client_negotiation.cc:240] Negotiation successful
0806 12:31:26.463827 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7437,"thread_start_us":117,"threads_started":1}
I20260806 12:31:26.465740 20451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027ca27c857c49718f311c810caacb08" candidate_uuid: "f7fc075232ef4ece8867a25aa80b47b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7c5497fc3c4471b49a0b4689496fba"
I20260806 12:31:26.465816 20433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "027ca27c857c49718f311c810caacb08" candidate_uuid: "f7fc075232ef4ece8867a25aa80b47b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7c5497fc3c4471b49a0b4689496fba" is_pre_election: true
I20260806 12:31:26.465992 20451 raft_consensus.cc:3037] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:26.468150 20451 raft_consensus.cc:2454] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7fc075232ef4ece8867a25aa80b47b2 in term 1.
I20260806 12:31:26.522787 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.516873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49528 (local address 127.6.152.131:39667)
0806 12:31:26.517050 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:26.517055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:26.517071 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:31:26.517163 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:26.517166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:26.517229 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:26.517367 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:31:26.517653 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.518633 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.520107 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:26.520452 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:26.520580 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:26.521283 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:31:26.521910 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:31:26.522041 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:26.522554 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 12:31:26.522620 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 12:31:26.531603 20451 raft_consensus.cc:1261] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 1 FOLLOWER]: Refusing update from remote peer f7fc075232ef4ece8867a25aa80b47b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:26.531747 20296 raft_consensus.cc:1261] T 027ca27c857c49718f311c810caacb08 P 694a7004ccf84871a3a21169502cd816 [term 1 FOLLOWER]: Refusing update from remote peer f7fc075232ef4ece8867a25aa80b47b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:26.532361 20688 consensus_queue.cc:1035] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d7c5497fc3c4471b49a0b4689496fba" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 32983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:26.532835 20681 consensus_queue.cc:1035] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "694a7004ccf84871a3a21169502cd816" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:26.540755 20693 mvcc.cc:205] Tried to move back new op lower bound from 7315535816825204736 to 7315535816500285440. Current Snapshot: MvccSnapshot[applied={T|T < 7315535816825204736}]
I20260806 12:31:26.549048 20697 mvcc.cc:205] Tried to move back new op lower bound from 7315535816825204736 to 7315535816500285440. Current Snapshot: MvccSnapshot[applied={T|T < 7315535816825204736}]
I20260806 12:31:26.549738 20698 mvcc.cc:205] Tried to move back new op lower bound from 7315535816825204736 to 7315535816500285440. Current Snapshot: MvccSnapshot[applied={T|T < 7315535816825204736}]
I20260806 12:31:26.556257 20159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:26.556440 20159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:26.559064 20159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d47db764edd4b76aa7de2cd7b2b20d4: Sending DeleteTablet for 3 replicas of tablet 027ca27c857c49718f311c810caacb08
I20260806 12:31:26.559829 20294 tablet_service.cc:1514] Processing DeleteTablet for tablet 027ca27c857c49718f311c810caacb08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:26 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:41918
I20260806 12:31:26.560060 20413 tablet_service.cc:1514] Processing DeleteTablet for tablet 027ca27c857c49718f311c810caacb08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:26 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:48592
I20260806 12:31:26.561051 20570 tablet_service.cc:1514] Processing DeleteTablet for tablet 027ca27c857c49718f311c810caacb08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:26 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:49522
I20260806 12:31:26.561616 20710 tablet_replica.cc:330] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: stopping tablet replica
I20260806 12:31:26.561641 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20198
I20260806 12:31:26.562013 20710 raft_consensus.cc:2229] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:31:26.562549 20710 raft_consensus.cc:2258] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:26.564113 20710 ts_tablet_manager.cc:1905] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:31:26.567377 20710 ts_tablet_manager.cc:1918] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:26.567543 20710 log.cc:1196] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/027ca27c857c49718f311c810caacb08
I20260806 12:31:26.568046 20710 ts_tablet_manager.cc:1939] T 027ca27c857c49718f311c810caacb08 P f7fc075232ef4ece8867a25aa80b47b2: Deleting consensus metadata
I20260806 12:31:26.568221 20709 tablet_replica.cc:330] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: stopping tablet replica
I20260806 12:31:26.569958 20145 catalog_manager.cc:4841] TS f7fc075232ef4ece8867a25aa80b47b2 (127.6.152.131:39667): tablet 027ca27c857c49718f311c810caacb08 (table test-table [id=8f1bf5c6173d465eb26245df5d713b00]) successfully deleted
I20260806 12:31:26.570164 20709 raft_consensus.cc:2229] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:31:26.570487 20709 raft_consensus.cc:2258] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:26.574946 20709 ts_tablet_manager.cc:1905] T 027ca27c857c49718f311c810caacb08 P 8d7c5497fc3c4471b49a0b4689496fba: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 12:31:26.575289 20144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:37067 (error 108)
I20260806 12:31:26.576179 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20364
I20260806 12:31:26.579552 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.576576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37067 (local address 127.0.0.1:41926)
0806 12:31:26.579327 (+ 2751us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.579420 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:37067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2653,"thread_start_us":52,"threads_started":1}
W20260806 12:31:26.579691 20144 catalog_manager.cc:4584] TS 694a7004ccf84871a3a21169502cd816 (127.6.152.129:37067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 027ca27c857c49718f311c810caacb08: Network error: Client connection negotiation failed: client connection to 127.6.152.129:37067: connect: Connection refused (error 111)
I20260806 12:31:26.582922 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20502
I20260806 12:31:26.585523 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:26.585189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:32983 (local address 127.0.0.1:48598)
0806 12:31:26.585358 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:26.585425 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.130:32983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 12:31:26.589808 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20125
I20260806 12:31:26.600121 6754 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 12:31:26.944828 6754 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 12:31:26.944944 6754 mini_ranger.cc:66] Stopping Ranger...
I20260806 12:31:27.289327 6754 mini_ranger.cc:68] Stopped Ranger
2026-08-06 12:31:27.289 UTC [19164] LOG: received smart shutdown request
2026-08-06 12:31:27.294 UTC [19164] LOG: background worker "logical replication launcher" (PID 19174) exited with exit code 1
2026-08-06 12:31:27.294 UTC [19168] LOG: shutting down
2026-08-06 12:31:27.459 UTC [19164] LOG: database system is shut down
2026-08-06T12:31:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48351 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:31:27 dist-test-slave-1xrh krb5kdc[20722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:31:27 dist-test-slave-1xrh krb5kdc[20722](info): set up 2 sockets
Aug 06 12:31:27 dist-test-slave-1xrh krb5kdc[20722](info): commencing operation
krb5kdc: starting...
W20260806 12:31:28.645890 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:31:28 dist-test-slave-1xrh krb5kdc[20722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:31:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:31:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:31:28.769987 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:41189
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:41189,127.6.152.189:46495,127.6.152.188:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:28.878650 20735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:28.878886 20735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:28.878934 20735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:28.882740 20735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:28.882787 20735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:28.882807 20735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:28.882823 20735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:28.882838 20735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:28.887851 20735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:41189,127.6.152.189:46495,127.6.152.188:44989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:41189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:28.888702 20735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:28.889586 20735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:28.897745 20741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:28.897693 20740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:28.897694 20743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:28.898105 20735 server_base.cc:1034] running on GCE node
I20260806 12:31:28.898674 20735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:28.899750 20735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:28.900929 20735 hybrid_clock.cc:648] HybridClock initialized: now 1786019488900907 us; error 42 us; skew 500 ppm
Aug 06 12:31:28 dist-test-slave-1xrh krb5kdc[20722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:28.903630 20735 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:28.904467 20735 webserver.cc:460] Webserver started at http://127.6.152.190:38203/ using document root <none> and password file <none>
I20260806 12:31:28.905001 20735 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:28.905093 20735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:28.905339 20735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:28.907399 20735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "0fcc4471d9384a9d9694292319978a81"
format_stamp: "Formatted at 2026-08-06 12:31:28 on dist-test-slave-1xrh"
server_key: "f4908d6e3ab3d6f1ff5e4ade6ec45ea1"
server_key_iv: "479bec3daea2ad760bd8ec98557034e3"
server_key_version: "encryptionkey@0"
I20260806 12:31:28.908054 20735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "0fcc4471d9384a9d9694292319978a81"
format_stamp: "Formatted at 2026-08-06 12:31:28 on dist-test-slave-1xrh"
server_key: "f4908d6e3ab3d6f1ff5e4ade6ec45ea1"
server_key_iv: "479bec3daea2ad760bd8ec98557034e3"
server_key_version: "encryptionkey@0"
I20260806 12:31:28.911464 20735 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:31:28.913720 20751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:28.914772 20735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:28.914927 20735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "0fcc4471d9384a9d9694292319978a81"
format_stamp: "Formatted at 2026-08-06 12:31:28 on dist-test-slave-1xrh"
server_key: "f4908d6e3ab3d6f1ff5e4ade6ec45ea1"
server_key_iv: "479bec3daea2ad760bd8ec98557034e3"
server_key_version: "encryptionkey@0"
I20260806 12:31:28.915033 20735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:28.932562 20735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:28.935231 20735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:28.935374 20735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:28.942132 20735 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:41189
I20260806 12:31:28.942164 20806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:41189 every 8 connection(s)
I20260806 12:31:28.943010 20735 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:28.945642 20807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:28.946105 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20735
I20260806 12:31:28.946264 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:31:28.946588 6754 external_mini_cluster.cc:1428] Setting key debaa7441099fcdbd57460f444ee748b
I20260806 12:31:28.950249 20807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.152.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.189@KRBTEST.COM" created.
Aug 06 12:31:28 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:28.965183 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:28.952583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:41189 (local address 127.0.0.1:48810)
0806 12:31:28.953128 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:28.953163 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:28.953984 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:28.954641 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:28.954646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:28.955070 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:28.955356 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:28.955369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:28.956587 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:28.956590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:28.957333 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:28.957428 (+ 95us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:28.958119 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:28.958122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:28.958308 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:28.958877 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:28.958893 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:28.960292 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:28.962217 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:28.962227 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:28.962236 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:28.962469 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:28.962792 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:28.962796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:28.962798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:28.962915 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:28.963541 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:28.963546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:28.963689 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 12:31:28.963869 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 12:31:28.964252 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":160,"thread_start_us":150,"threads_started":1}
I20260806 12:31:28.965188 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:28.952582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.6.152.190:41189)
0806 12:31:28.952974 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:28.952994 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:28.953287 (+ 293us) server_negotiation.cc:407] Connection header received
0806 12:31:28.954281 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:28.954285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:28.954461 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:28.954642 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:28.955505 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:28.956448 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:28.957562 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:28.957923 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:28.958101 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:28.960453 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:28.960459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:28.960467 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:28.960487 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:28.962088 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:28.962581 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:28.962587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:28.962589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:28.962694 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:28.963016 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:28.963019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:28.963021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:28.963427 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:28.963569 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:28.963932 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 12:31:28.964199 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":255,"thread_start_us":131,"threads_started":1}
I20260806 12:31:28.967533 20807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:31:28.968247 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:28.967995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37276)
0806 12:31:28.968113 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:28.968185 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 12:31:28.968451 20753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
I20260806 12:31:28.969363 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:28.969153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37284)
0806 12:31:28.969266 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:28.969310 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 12:31:28.969523 20807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:46495: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab.
Entry for principal kudu/127.6.152.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab.
I20260806 12:31:28.973503 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:46495
--webserver_interface=127.6.152.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab
--principal=kudu/127.6.152.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:41189,127.6.152.189:46495,127.6.152.188:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:29.018769 20807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } attempt: 1
I20260806 12:31:29.019838 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.019478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37292)
0806 12:31:29.019690 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.019756 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 12:31:29.020573 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.020338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37300)
0806 12:31:29.020446 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.020497 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 12:31:29.020736 20807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:46495: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
W20260806 12:31:29.082350 20814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:29.082628 20814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:29.082708 20814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:29.086611 20814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:29.086696 20814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:29.086733 20814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:29.086783 20814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:29.086826 20814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:29.088886 20807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } attempt: 2
I20260806 12:31:29.089733 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.089419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37310)
0806 12:31:29.089582 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.089654 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 12:31:29.090448 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.090237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37326)
0806 12:31:29.090344 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.090396 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 12:31:29.090602 20807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:46495: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46495: connect: Connection refused (error 111)
I20260806 12:31:29.092231 20814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:41189,127.6.152.189:46495,127.6.152.188:44989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab
--principal=kudu/127.6.152.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:46495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.152.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:29.093189 20814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:29.094190 20814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:29.102348 20819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:29.102353 20820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:29.102353 20822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:29.102532 20814 server_base.cc:1034] running on GCE node
I20260806 12:31:29.103243 20814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:29.104293 20814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:29.105484 20814 hybrid_clock.cc:648] HybridClock initialized: now 1786019489105466 us; error 26 us; skew 500 ppm
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:29.108345 20814 init.cc:377] Logged in from keytab as kudu/127.6.152.189@KRBTEST.COM (short username kudu)
I20260806 12:31:29.109169 20814 webserver.cc:460] Webserver started at http://127.6.152.189:46327/ using document root <none> and password file <none>
I20260806 12:31:29.109658 20814 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:29.109714 20814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:29.109892 20814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:29.111855 20814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data/instance:
uuid: "25d84b238c35461291be3207e51acdf6"
format_stamp: "Formatted at 2026-08-06 12:31:29 on dist-test-slave-1xrh"
server_key: "d6efd1f7b31c53ef3bc29254b86a0d4a"
server_key_iv: "dceab91c9ff4051bbbb20b8bab6c7001"
server_key_version: "encryptionkey@0"
I20260806 12:31:29.112406 20814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal/instance:
uuid: "25d84b238c35461291be3207e51acdf6"
format_stamp: "Formatted at 2026-08-06 12:31:29 on dist-test-slave-1xrh"
server_key: "d6efd1f7b31c53ef3bc29254b86a0d4a"
server_key_iv: "dceab91c9ff4051bbbb20b8bab6c7001"
server_key_version: "encryptionkey@0"
I20260806 12:31:29.116061 20814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:31:29.118479 20829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:29.119889 20814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:31:29.120025 20814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal
uuid: "25d84b238c35461291be3207e51acdf6"
format_stamp: "Formatted at 2026-08-06 12:31:29 on dist-test-slave-1xrh"
server_key: "d6efd1f7b31c53ef3bc29254b86a0d4a"
server_key_iv: "dceab91c9ff4051bbbb20b8bab6c7001"
server_key_version: "encryptionkey@0"
I20260806 12:31:29.120144 20814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:29.131270 20814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:29.134938 20814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:29.135181 20814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:29.142131 20814 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.189:46495
I20260806 12:31:29.142242 20882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.189:46495 every 8 connection(s)
I20260806 12:31:29.143043 20814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
I20260806 12:31:29.145634 20883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:29.149122 20883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:31:29.149742 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20814
I20260806 12:31:29.149924 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-1/wal/instance
I20260806 12:31:29.150197 6754 external_mini_cluster.cc:1428] Setting key fcc5fbdd993679c511e8b87e92402760
WARNING: no policy specified for kudu/127.6.152.188@KRBTEST.COM; defaulting to no policy
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.188@KRBTEST.COM" created.
I20260806 12:31:29.163668 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.150978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48824 (local address 127.6.152.190:41189)
0806 12:31:29.151177 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.151185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.151965 (+ 780us) server_negotiation.cc:407] Connection header received
0806 12:31:29.152931 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.152936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.153003 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.153114 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.154181 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.155077 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.156187 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.156605 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.156744 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.159323 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.159326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.159329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.159354 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.161410 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.161985 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.161991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.161995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.162096 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.162438 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.162445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.162450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.162734 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.162846 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.163367 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.163496 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 12:31:29.164505 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.151367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:41189 (local address 127.0.0.1:48824)
0806 12:31:29.151817 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.151868 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.152721 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.153255 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.153261 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.153583 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.154013 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.154026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.155228 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.155231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.156055 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.156064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.156775 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.156777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.156911 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.157491 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.157507 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.159094 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.161524 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.161533 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.161549 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.161883 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.162198 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.162201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.162202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.162342 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.162844 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.162848 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.163065 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.163311 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.163610 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":139,"threads_started":1}
I20260806 12:31:29.165983 20883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:31:29.176476 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.166499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37342)
0806 12:31:29.166697 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.166713 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.166793 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.167241 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.167246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.167438 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.167576 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.167588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.168536 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.168542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.169477 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.169490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.170086 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.170090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.170199 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.170629 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.170640 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.172921 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.174888 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.174891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.174892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.175201 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.175484 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.175489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.175491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.175637 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.176106 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.176108 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.176187 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.176273 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.176368 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 12:31:29.176764 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.166580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.6.152.189:46495)
0806 12:31:29.166843 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.166855 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.166879 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:31:29.166919 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.166923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.167059 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.167217 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.167712 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.168406 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.169614 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.169953 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.170030 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.173031 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.173039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.173051 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.173072 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.174777 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.175282 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.175286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.175287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.175366 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.175752 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.175754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.175755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.175978 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.176146 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.176396 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.176530 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab.
Entry for principal kudu/127.6.152.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab.
I20260806 12:31:29.178452 20883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:31:29.178678 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:44989
--webserver_interface=127.6.152.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab
--principal=kudu/127.6.152.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:41189,127.6.152.189:46495,127.6.152.188:44989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:29.180157 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.178898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42276)
0806 12:31:29.180040 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.180106 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1074}
W20260806 12:31:29.180439 20831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
I20260806 12:31:29.181346 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.181151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42292)
0806 12:31:29.181257 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.181300 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 12:31:29.181577 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:44989: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
I20260806 12:31:29.181766 20807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } attempt: 3
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:31:29.191419 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.182197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:37344)
0806 12:31:29.182358 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.182376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.182477 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.182883 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.182885 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.183024 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.183183 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.183192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.184753 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.184756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.185284 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.185291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.185980 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.185983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.186087 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.186606 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.186617 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.188170 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.189809 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.189816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.189821 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.190113 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.190386 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.190392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.190397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.190518 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.190874 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.190877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.190983 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.191094 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.191250 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 12:31:29.191430 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.182292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.6.152.189:46495)
0806 12:31:29.182643 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.182650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.182665 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:31:29.182703 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.182709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.182762 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.182848 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.183315 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.184639 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.185401 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.185867 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.185994 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.188306 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.188310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.188312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.188332 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.189683 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.190221 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.190223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.190224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.190292 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.190625 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.190627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.190628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.190781 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.190853 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.191243 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.191335 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260806 12:31:29.192365 20807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:31:29.193053 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.192792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42306)
0806 12:31:29.192935 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.192982 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 12:31:29.193859 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.193624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42320)
0806 12:31:29.193767 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.193809 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 12:31:29.194054 20807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:44989: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
I20260806 12:31:29.225217 20807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } attempt: 1
I20260806 12:31:29.226096 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.225783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42326)
0806 12:31:29.225940 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.226006 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 12:31:29.226821 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.226625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42332)
0806 12:31:29.226727 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.226769 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 12:31:29.226991 20807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:44989: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
I20260806 12:31:29.230813 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } attempt: 1
I20260806 12:31:29.231513 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.231273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42340)
0806 12:31:29.231397 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.231448 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 12:31:29.232223 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.232022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42344)
0806 12:31:29.232126 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.232169 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 12:31:29.232424 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:44989: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
W20260806 12:31:29.291841 20889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:29.292076 20889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:29.292116 20889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:29.296092 20889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:29.296140 20889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:29.296160 20889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:29.296201 20889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:29.296233 20889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:29.300588 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } attempt: 2
I20260806 12:31:29.301501 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.301202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42356)
0806 12:31:29.301362 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.301424 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 12:31:29.301546 20889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:41189,127.6.152.189:46495,127.6.152.188:44989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab
--principal=kudu/127.6.152.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:44989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.152.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:29.302179 20807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } attempt: 2
I20260806 12:31:29.302449 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.302196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42364)
0806 12:31:29.302320 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.302375 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 12:31:29.302723 20889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260806 12:31:29.302778 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:44989: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
I20260806 12:31:29.302884 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.302590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42380)
0806 12:31:29.302718 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.302796 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 12:31:29.303634 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.303447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:42392)
0806 12:31:29.303543 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.303585 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 12:31:29.303849 20807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:44989: Network error: Client connection negotiation failed: client connection to 127.6.152.188:44989: connect: Connection refused (error 111)
I20260806 12:31:29.304064 20889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:29.312186 20899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:29.312391 20896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:29.316941 20897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:29.317286 20889 server_base.cc:1034] running on GCE node
I20260806 12:31:29.318317 20889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:29.319900 20889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:29.321022 20889 hybrid_clock.cc:648] HybridClock initialized: now 1786019489321005 us; error 31 us; skew 500 ppm
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:29.324817 20889 init.cc:377] Logged in from keytab as kudu/127.6.152.188@KRBTEST.COM (short username kudu)
I20260806 12:31:29.325762 20889 webserver.cc:460] Webserver started at http://127.6.152.188:32895/ using document root <none> and password file <none>
I20260806 12:31:29.326397 20889 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:29.326502 20889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:29.326767 20889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:29.329051 20889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data/instance:
uuid: "c55f39c2069f4bd68b7d5fc36651f462"
format_stamp: "Formatted at 2026-08-06 12:31:29 on dist-test-slave-1xrh"
server_key: "543f3da188df9d0c2d285e1a6e75d8b2"
server_key_iv: "0e5c4b490212b81bc4062de216254733"
server_key_version: "encryptionkey@0"
I20260806 12:31:29.329684 20889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal/instance:
uuid: "c55f39c2069f4bd68b7d5fc36651f462"
format_stamp: "Formatted at 2026-08-06 12:31:29 on dist-test-slave-1xrh"
server_key: "543f3da188df9d0c2d285e1a6e75d8b2"
server_key_iv: "0e5c4b490212b81bc4062de216254733"
server_key_version: "encryptionkey@0"
I20260806 12:31:29.333664 20889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 12:31:29.336164 20906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:29.337285 20889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:29.337426 20889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal
uuid: "c55f39c2069f4bd68b7d5fc36651f462"
format_stamp: "Formatted at 2026-08-06 12:31:29 on dist-test-slave-1xrh"
server_key: "543f3da188df9d0c2d285e1a6e75d8b2"
server_key_iv: "0e5c4b490212b81bc4062de216254733"
server_key_version: "encryptionkey@0"
I20260806 12:31:29.337554 20889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:29.352208 20889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:29.355922 20889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:29.356134 20889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:29.363489 20889 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.188:44989
I20260806 12:31:29.363525 20961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.188:44989 every 8 connection(s)
I20260806 12:31:29.364451 20889 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
I20260806 12:31:29.365473 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 20889
I20260806 12:31:29.365639 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786019324359312-6754-0/minicluster-data/master-2/wal/instance
I20260806 12:31:29.365937 6754 external_mini_cluster.cc:1428] Setting key 7e15178ba2f5b72607027430445ff298
I20260806 12:31:29.367554 20962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:29.372116 20962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:29.380884 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.367326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.6.152.188:44989)
0806 12:31:29.367738 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.367756 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.367796 (+ 40us) server_negotiation.cc:407] Connection header received
0806 12:31:29.368637 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.368659 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.369063 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.369419 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.370082 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.371580 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.372423 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.373067 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.373223 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.376132 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.376137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.376145 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.376170 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.378002 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.378488 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.378495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.378501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.378655 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.378948 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.378951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.378953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.379477 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.379718 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.379952 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.380214 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260806 12:31:29.383988 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.373946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:41189 (local address 127.0.0.1:57560)
0806 12:31:29.374485 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.374544 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.374674 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.374988 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.374992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.375375 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.375503 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.375509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.376492 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.376494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.377516 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.377528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.377983 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.377987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.378118 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.378609 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.378623 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.380170 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.382093 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.382100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.382104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.382311 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.382720 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.382722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.382723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.382832 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.383379 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.383385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.383510 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.383657 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.383849 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":146,"threads_started":1}
I20260806 12:31:29.384307 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.374021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.6.152.190:41189)
0806 12:31:29.374201 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.374205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.374639 (+ 434us) server_negotiation.cc:407] Connection header received
0806 12:31:29.374778 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.374781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.374825 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.374886 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.375707 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.376345 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.377649 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.377857 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.377935 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.380314 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.380317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.380319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.380342 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.381970 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.382413 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.382420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.382423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.382508 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.382967 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.382972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.382977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.383243 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.383338 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.383732 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.383877 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 12:31:29.385270 20962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:31:29.392639 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.383082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.6.152.190:41189)
0806 12:31:29.383463 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.383473 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.383487 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:31:29.383523 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.383525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.383568 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.383662 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.384441 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.385183 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.386425 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.386785 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.386890 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.389434 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.389438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.389440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.389471 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.390964 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.391424 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.391428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.391429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.391516 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.391803 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.391806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.391808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.391982 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.392171 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.392353 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.392486 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":132,"threads_started":1}
I20260806 12:31:29.394033 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } attempt: 3
I20260806 12:31:29.394186 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.385721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46495 (local address 127.0.0.1:52828)
0806 12:31:29.385855 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.385865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.385934 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.386151 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.386153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.386276 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.386402 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.386408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.387369 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.387372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.388333 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.388346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.388801 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.388805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.388906 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.389340 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.389360 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.390723 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.392443 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.392448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.392450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.392735 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.393059 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.393062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.393064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.393162 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.393692 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.393695 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.393797 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.393905 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.394032 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 12:31:29.394256 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.385838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.6.152.189:46495)
0806 12:31:29.385956 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.385960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.385973 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:31:29.386010 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.386013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.386056 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.386123 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.386532 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.387259 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.388464 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.388676 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.388798 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.390862 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.390866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.390869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.390898 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.392326 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.392845 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.392849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.392851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.392947 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.393271 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.393277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.393282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.393508 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.393609 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.393974 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.394121 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:31:29.395404 20962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:31:29.403167 20807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } attempt: 3
I20260806 12:31:29.404137 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.393827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.6.152.189:46495)
0806 12:31:29.394407 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.394413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.394427 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:31:29.394459 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.394461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.394502 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.394580 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.395796 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.396438 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.397363 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.397660 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.397759 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.400640 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.400644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.400646 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.400670 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.402032 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.402520 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.402523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.402524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.402602 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.402944 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.402949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.402951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.403249 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.403468 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.403879 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.403999 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":527,"spinlock_wait_cycles":3328,"thread_start_us":122,"threads_started":1}
I20260806 12:31:29.404412 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.394395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:47154)
0806 12:31:29.394515 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.394527 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.394592 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.395058 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.395061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.395255 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.395326 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.395353 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.396623 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.396629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.397210 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.397219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.397755 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.397759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.397929 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.398252 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.398261 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.399505 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.402576 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.402579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.402581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.402863 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.403173 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.403176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.403178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.403279 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.403866 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.403872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.403978 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.404080 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.404216 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":70,"spinlock_wait_cycles":3456}
I20260806 12:31:29.405496 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.395863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:47164)
0806 12:31:29.396040 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.396053 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.396117 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.396510 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.396516 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.396680 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.396757 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.396763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.397658 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.397664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.398822 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.398837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.399539 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.399544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.399631 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.400081 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.400100 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.401598 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.403937 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.403940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.403941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.404170 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.404455 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.404459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.404460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.404568 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.404999 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.405002 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.405112 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.405214 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.405334 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":1,"spinlock_wait_cycles":3456}
I20260806 12:31:29.405637 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.395957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.6.152.188:44989)
0806 12:31:29.396313 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.396321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.396331 (+ 10us) server_negotiation.cc:407] Connection header received
0806 12:31:29.396357 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.396359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.396390 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.396475 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.396877 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.397559 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.398965 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.399378 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.399521 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.401741 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.401746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.401751 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.401777 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.403807 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.404273 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.404276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.404278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.404348 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.404682 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.404686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.404689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.404899 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.405010 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.405270 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.405450 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":304,"thread_start_us":104,"threads_started":1}
I20260806 12:31:29.406497 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.394718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.6.152.188:44989)
0806 12:31:29.394849 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.394853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.394865 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:31:29.394895 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.394897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.394935 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.395020 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.395452 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.396495 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.397325 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.397657 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.397759 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.399630 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.399634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.399636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.399672 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.402452 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.402956 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.402960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.402962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.403044 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.403475 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.403479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.403481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.403689 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.403782 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.406219 (+ 2437us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.406347 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":45}
I20260806 12:31:29.408957 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:31:29.409106 20962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: Bootstrap starting.
Aug 06 12:31:29 dist-test-slave-1xrh krb5kdc[20722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:31:29.409626 20883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: Bootstrap starting.
I20260806 12:31:29.412056 20962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:29.412249 20883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:29.412995 20962 log.cc:824] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:29.413120 20883 log.cc:824] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:29.413815 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.403545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:44989 (local address 127.0.0.1:47180)
0806 12:31:29.403713 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:29.403728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:29.403817 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:29.404840 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:29.404843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:29.405027 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:29.405137 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.405143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.406350 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.406354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.407179 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:29.407190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.407966 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.407969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:29.408045 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.408482 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:29.408496 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:29.409867 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:29.412107 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.412113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.412118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.412389 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.412734 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:29.412737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:29.412739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:29.412829 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:29.413293 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:29.413297 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:29.413410 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 12:31:29.413497 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 12:31:29.413630 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 12:31:29.414395 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.404286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.6.152.188:44989)
0806 12:31:29.404571 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.404583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.404598 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:31:29.404633 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.404637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.404691 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.405250 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.405422 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.406217 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.407474 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.407818 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.408241 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.410033 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.410037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.410039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.410062 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.411974 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.412537 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.412541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.412543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.412624 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.412963 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.412966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.412967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.413167 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.414011 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.414087 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.414219 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":214,"thread_start_us":118,"threads_started":1}
I20260806 12:31:29.415761 20883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: No bootstrap required, opened a new log
I20260806 12:31:29.417762 20807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81: Bootstrap starting.
I20260806 12:31:29.420056 20883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.420310 20883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:29.420369 20883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d84b238c35461291be3207e51acdf6, State: Initialized, Role: FOLLOWER
I20260806 12:31:29.421165 20883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.421901 20807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:29.422406 20962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: No bootstrap required, opened a new log
I20260806 12:31:29.422470 20883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:29.422765 20807 log.cc:824] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:29.425251 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.412492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.6.152.188:44989)
0806 12:31:29.413505 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.413509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.415051 (+ 1542us) server_negotiation.cc:407] Connection header received
0806 12:31:29.415202 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.415205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.415249 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.415329 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.416308 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.417015 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.418229 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.418528 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.418613 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.420549 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.420553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.420555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.420580 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.422358 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.423014 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.423018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.423020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.423149 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.423465 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.423470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.423472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.424037 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.424170 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.424957 (+ 787us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.425083 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932}
I20260806 12:31:29.426364 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.411837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.6.152.189:46495)
0806 12:31:29.411955 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.411959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.414709 (+ 2750us) server_negotiation.cc:407] Connection header received
0806 12:31:29.415244 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.415248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.415291 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.415341 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.416553 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.417345 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.418493 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.418814 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.418915 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.420787 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.420792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.420797 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.420825 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.424504 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.424985 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.424988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.424990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.425074 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.425410 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.425415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.425418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.425648 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.425747 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.426114 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.426226 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:31:29.427543 20962 raft_consensus.cc:348] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.427794 20962 raft_consensus.cc:374] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:29.427856 20962 raft_consensus.cc:729] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55f39c2069f4bd68b7d5fc36651f462, State: Initialized, Role: FOLLOWER
I20260806 12:31:29.428330 20962 consensus_queue.cc:260] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.428864 20993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.428997 20993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.429652 20962 sys_catalog.cc:564] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:29.426060 20814 master_runner.cc:186] Error getting master registration for 127.6.152.190:41189: OK, instance_id { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" instance_seqno: 1786019488940202 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:31:29.430313 20814 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 12:31:29.434856 20992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:31:29.434934 20992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 12:31:29.434904 21004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:31:29.435060 21004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:31:29.435142 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.435271 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.433151 20889 master_runner.cc:186] Error getting master registration for 127.6.152.190:41189: OK, instance_id { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" instance_seqno: 1786019488940202 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:31:29.436228 20889 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:31:29.438973 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:29.413062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.6.152.190:41189)
0806 12:31:29.414110 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:29.414115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:29.420644 (+ 6529us) server_negotiation.cc:407] Connection header received
0806 12:31:29.420687 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:29.420693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:29.420744 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:29.420817 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:29.431416 (+ 10599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.432185 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.433278 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:29.433607 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:29.433715 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:29.435754 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:29.435758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:29.435760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:29.435787 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:29.437320 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.437791 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.437794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.437795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.437869 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:29.438168 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:29.438170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:29.438171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:29.438329 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:29.438432 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:29.438715 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 12:31:29.438808 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749}
I20260806 12:31:29.440984 20807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81: No bootstrap required, opened a new log
I20260806 12:31:29.445086 20807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.445300 20807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:29.445355 20807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcc4471d9384a9d9694292319978a81, State: Initialized, Role: FOLLOWER
I20260806 12:31:29.445806 20807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.447177 20807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:29.447409 21007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.447602 21007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.450153 21007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:29.450281 21007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.451200 21007 leader_election.cc:288] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25d84b238c35461291be3207e51acdf6 (127.6.152.189:46495), c55f39c2069f4bd68b7d5fc36651f462 (127.6.152.188:44989)
W20260806 12:31:29.452785 21018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:31:29.452925 21018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:31:29.453178 20857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fcc4471d9384a9d9694292319978a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25d84b238c35461291be3207e51acdf6" is_pre_election: true
I20260806 12:31:29.453454 20857 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fcc4471d9384a9d9694292319978a81 in term 0.
I20260806 12:31:29.453877 20753 leader_election.cc:302] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fcc4471d9384a9d9694292319978a81, 25d84b238c35461291be3207e51acdf6; no voters:
I20260806 12:31:29.453897 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fcc4471d9384a9d9694292319978a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55f39c2069f4bd68b7d5fc36651f462" is_pre_election: true
I20260806 12:31:29.454174 20935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fcc4471d9384a9d9694292319978a81 in term 0.
I20260806 12:31:29.455127 21007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:31:29.455241 21007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:31:29.455297 21007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:29.456692 21007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.456990 21007 leader_election.cc:288] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [CANDIDATE]: Term 1 election: Requested vote from peers 25d84b238c35461291be3207e51acdf6 (127.6.152.189:46495), c55f39c2069f4bd68b7d5fc36651f462 (127.6.152.188:44989)
I20260806 12:31:29.458083 20935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fcc4471d9384a9d9694292319978a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55f39c2069f4bd68b7d5fc36651f462"
I20260806 12:31:29.458197 20935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:29.459453 20935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fcc4471d9384a9d9694292319978a81 in term 1.
I20260806 12:31:29.459909 20857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0fcc4471d9384a9d9694292319978a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25d84b238c35461291be3207e51acdf6"
I20260806 12:31:29.460057 20857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:29.461305 20857 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fcc4471d9384a9d9694292319978a81 in term 1.
I20260806 12:31:29.461678 20753 leader_election.cc:302] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fcc4471d9384a9d9694292319978a81, 25d84b238c35461291be3207e51acdf6; no voters:
I20260806 12:31:29.462182 21007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:29.462405 21007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [term 1 LEADER]: Becoming Leader. State: Replica: 0fcc4471d9384a9d9694292319978a81, State: Running, Role: LEADER
I20260806 12:31:29.462749 21007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } }
I20260806 12:31:29.463323 20735 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260806 12:31:29.463418 20735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:31:29.464331 21007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcc4471d9384a9d9694292319978a81. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.464437 21007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:29.471436 21022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:29.472483 21022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:29.478751 20935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [term 1 FOLLOWER]: Refusing update from remote peer 0fcc4471d9384a9d9694292319978a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:29.479524 21020 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:29.483424 20993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcc4471d9384a9d9694292319978a81. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.483551 20993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.484683 20857 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [term 1 FOLLOWER]: Refusing update from remote peer 0fcc4471d9384a9d9694292319978a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:29.485303 21020 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:29.489078 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcc4471d9384a9d9694292319978a81. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.489192 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.494675 21007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.494793 21007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:29.495232 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.495342 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.496649 21007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.496760 21007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:29.497417 21023 mvcc.cc:205] Tried to move back new op lower bound from 7315535828897677312 to 7315535828842516480. Current Snapshot: MvccSnapshot[applied={T|T < 7315535828897677312}]
I20260806 12:31:29.497934 20993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fcc4471d9384a9d9694292319978a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fcc4471d9384a9d9694292319978a81" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 41189 } } peers { permanent_uuid: "25d84b238c35461291be3207e51acdf6" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46495 } } peers { permanent_uuid: "c55f39c2069f4bd68b7d5fc36651f462" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 44989 } } }
I20260806 12:31:29.498044 20993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:31:29.498801 21024 mvcc.cc:205] Tried to move back new op lower bound from 7315535828897677312 to 7315535828842516480. Current Snapshot: MvccSnapshot[applied={T|T < 7315535828897677312}]
I20260806 12:31:29.499243 21026 mvcc.cc:205] Tried to move back new op lower bound from 7315535828897677312 to 7315535828842516480. Current Snapshot: MvccSnapshot[applied={T|T < 7315535828897677312}]
I20260806 12:31:29.502249 21022 catalog_manager.cc:1347] Generated new cluster ID: e3495da51f064e57a8ef8b3ea2ecfbb1
I20260806 12:31:29.502305 21022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:29.524899 21022 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:31:29.526130 21022 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:29.546890 21022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fcc4471d9384a9d9694292319978a81: Generated new TSK 0
I20260806 12:31:29.547797 21022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:31:30.436398 20992 catalog_manager.cc:1259] Loaded cluster ID: e3495da51f064e57a8ef8b3ea2ecfbb1
I20260806 12:31:30.436398 21004 catalog_manager.cc:1259] Loaded cluster ID: e3495da51f064e57a8ef8b3ea2ecfbb1
I20260806 12:31:30.436484 20992 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: loading cluster ID for follower catalog manager: success
I20260806 12:31:30.436513 21004 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: loading cluster ID for follower catalog manager: success
I20260806 12:31:30.438663 21004 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: acquiring CA information for follower catalog manager: success
I20260806 12:31:30.438728 20992 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: acquiring CA information for follower catalog manager: success
I20260806 12:31:30.439832 20992 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25d84b238c35461291be3207e51acdf6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:31:30.439831 21004 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c55f39c2069f4bd68b7d5fc36651f462: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:31:30.621104 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20735
I20260806 12:31:30.629616 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20814
I20260806 12:31:30.636498 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 20889
2026-08-06T12:31:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3058 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:31:30 dist-test-slave-1xrh krb5kdc[21038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:31:30 dist-test-slave-1xrh krb5kdc[21038](info): set up 2 sockets
Aug 06 12:31:30 dist-test-slave-1xrh krb5kdc[21038](info): commencing operation
krb5kdc: starting...
W20260806 12:31:31.739296 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:31:31 dist-test-slave-1xrh krb5kdc[21038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:31:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:31:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:31:31.860320 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38603
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:38603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:31.968510 21051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:31.968796 21051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:31.968874 21051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:31.972822 21051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:31.972916 21051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:31.972961 21051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:31.972998 21051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:31.973032 21051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:31.978739 21051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:38603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:31.979830 21051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:31.980849 21051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:31.989313 21056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:31.989291 21057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:31.989286 21059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:31.989605 21051 server_base.cc:1034] running on GCE node
I20260806 12:31:31.990191 21051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:31.991305 21051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:31.992491 21051 hybrid_clock.cc:648] HybridClock initialized: now 1786019491992471 us; error 32 us; skew 500 ppm
Aug 06 12:31:31 dist-test-slave-1xrh krb5kdc[21038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:31.995215 21051 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:31.996068 21051 webserver.cc:460] Webserver started at http://127.6.152.190:43221/ using document root <none> and password file <none>
I20260806 12:31:31.996556 21051 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:31.996613 21051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:31.996768 21051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:31.998660 21051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d"
format_stamp: "Formatted at 2026-08-06 12:31:31 on dist-test-slave-1xrh"
server_key: "88fb9865b1328cbe5711181dee02f7de"
server_key_iv: "eeb53881f3b3cc87bc2c6a892a58124c"
server_key_version: "encryptionkey@0"
I20260806 12:31:31.999230 21051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d"
format_stamp: "Formatted at 2026-08-06 12:31:31 on dist-test-slave-1xrh"
server_key: "88fb9865b1328cbe5711181dee02f7de"
server_key_iv: "eeb53881f3b3cc87bc2c6a892a58124c"
server_key_version: "encryptionkey@0"
I20260806 12:31:32.002723 21051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:31:32.005097 21067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:32.006346 21051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:32.006486 21051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d"
format_stamp: "Formatted at 2026-08-06 12:31:31 on dist-test-slave-1xrh"
server_key: "88fb9865b1328cbe5711181dee02f7de"
server_key_iv: "eeb53881f3b3cc87bc2c6a892a58124c"
server_key_version: "encryptionkey@0"
I20260806 12:31:32.006598 21051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:32.022586 21051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:32.025401 21051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:32.025599 21051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:32.032681 21051 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:38603
I20260806 12:31:32.032711 21122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:38603 every 8 connection(s)
I20260806 12:31:32.033703 21051 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:32.035468 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21051
I20260806 12:31:32.035573 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:31:32.035887 6754 external_mini_cluster.cc:1428] Setting key a2d1b24f9b18a6947d3b3237c428ddf4
I20260806 12:31:32.036670 21123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:32.043534 21123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d: Bootstrap starting.
Aug 06 12:31:32 dist-test-slave-1xrh krb5kdc[21038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:32.045821 21123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:32.046833 21123 log.cc:824] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:32.049065 21123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d: No bootstrap required, opened a new log
I20260806 12:31:32.051015 21127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:32.037300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.6.152.190:38603)
0806 12:31:32.037727 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:32.037746 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:32.037787 (+ 41us) server_negotiation.cc:407] Connection header received
0806 12:31:32.038623 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:32.038650 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:32.039241 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:32.039616 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:32.040310 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:32.041783 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:32.042677 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:32.043379 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:32.043552 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:32.045963 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:32.045971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:32.045987 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:32.046019 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:32.048095 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:32.048523 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:32.048537 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:32.048550 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:32.048684 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:32.048941 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:32.048944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:32.048945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:32.049426 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:32.049621 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:32.049818 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 12:31:32.050098 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260806 12:31:32.053397 21123 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38603 } }
I20260806 12:31:32.053681 21123 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:32.053750 21123 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8ef72d1917f4a728ad73bc0dc9cb57d, State: Initialized, Role: FOLLOWER
I20260806 12:31:32.053957 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:31:32.054250 21123 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38603 } }
I20260806 12:31:32.054378 21123 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:32.054461 21123 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:32.054560 21123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:32.055691 21123 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38603 } }
I20260806 12:31:32.056043 21123 leader_election.cc:302] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8ef72d1917f4a728ad73bc0dc9cb57d; no voters:
I20260806 12:31:32.056326 21123 leader_election.cc:288] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:31:32.056452 21129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:32.056689 21129 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [term 1 LEADER]: Becoming Leader. State: Replica: b8ef72d1917f4a728ad73bc0dc9cb57d, State: Running, Role: LEADER
I20260806 12:31:32.057047 21129 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38603 } }
I20260806 12:31:32.057489 21123 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:32.058818 21131 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8ef72d1917f4a728ad73bc0dc9cb57d. Latest consensus state: current_term: 1 leader_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38603 } } }
I20260806 12:31:32.058847 21130 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ef72d1917f4a728ad73bc0dc9cb57d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38603 } } }
I20260806 12:31:32.058941 21131 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:32.058948 21130 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:32.059341 21138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:32.062016 21138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:32.067341 21138 catalog_manager.cc:1347] Generated new cluster ID: 8e156f666caa49c98cf30c9deddb3272
I20260806 12:31:32.067402 21138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:32.076048 21138 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:31:32.077162 21138 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:32.091744 21138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8ef72d1917f4a728ad73bc0dc9cb57d: Generated new TSK 0
I20260806 12:31:32.092489 21138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:31:32.109894 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21051
2026-08-06T12:31:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1471 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:31:32 dist-test-slave-1xrh krb5kdc[21152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:31:32 dist-test-slave-1xrh krb5kdc[21152](info): set up 2 sockets
Aug 06 12:31:32 dist-test-slave-1xrh krb5kdc[21152](info): commencing operation
krb5kdc: starting...
W20260806 12:31:33.187393 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:31:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:31:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:31:33.314951 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37149
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:33.432045 21165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:33.432301 21165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:33.432343 21165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:33.436321 21165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:33.436383 21165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:33.436403 21165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:33.436436 21165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:33.436452 21165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:33.441608 21165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:33.442658 21165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:33.443715 21165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:33.452204 21171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:33.452425 21165 server_base.cc:1034] running on GCE node
W20260806 12:31:33.452256 21170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:33.452205 21174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:33.453131 21165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:33.454210 21165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:33.455368 21165 hybrid_clock.cc:648] HybridClock initialized: now 1786019493455339 us; error 44 us; skew 500 ppm
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:33.465214 21165 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:33.466225 21165 webserver.cc:460] Webserver started at http://127.6.152.190:46459/ using document root <none> and password file <none>
I20260806 12:31:33.466795 21165 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:33.466898 21165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:33.467197 21165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:33.469267 21165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "55283a3fa83f4e6abb774d221ab41256"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "f1be9789f59dfb4ec9420849e0f1145a"
server_key_iv: "4a253202befd15525e112b3cd3006178"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.469882 21165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "55283a3fa83f4e6abb774d221ab41256"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "f1be9789f59dfb4ec9420849e0f1145a"
server_key_iv: "4a253202befd15525e112b3cd3006178"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.473654 21165 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:31:33.476208 21182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.477365 21165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:33.477504 21165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "55283a3fa83f4e6abb774d221ab41256"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "f1be9789f59dfb4ec9420849e0f1145a"
server_key_iv: "4a253202befd15525e112b3cd3006178"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.477617 21165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:33.492357 21165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:33.495574 21165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:33.495769 21165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:33.502802 21165 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37149
I20260806 12:31:33.503093 21238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37149 every 8 connection(s)
I20260806 12:31:33.503762 21165 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:33.506417 21239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:33.511574 21239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap starting.
I20260806 12:31:33.512981 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21165
I20260806 12:31:33.513089 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:31:33.513411 6754 external_mini_cluster.cc:1428] Setting key db94bda3dfb7d164e3682263cadb3e70
I20260806 12:31:33.514134 21239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:33.514912 21239 log.cc:824] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:33.516942 21239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: No bootstrap required, opened a new log
I20260806 12:31:33.520987 21239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:33.521255 21239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:33.521330 21239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55283a3fa83f4e6abb774d221ab41256, State: Initialized, Role: FOLLOWER
I20260806 12:31:33.521837 21239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:33.522027 21239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:33.522087 21239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:33.522151 21239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:33.523221 21239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:33.523543 21239 leader_election.cc:302] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55283a3fa83f4e6abb774d221ab41256; no voters:
I20260806 12:31:33.523844 21239 leader_election.cc:288] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:31:33.523957 21244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:33.524503 21244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 LEADER]: Becoming Leader. State: Replica: 55283a3fa83f4e6abb774d221ab41256, State: Running, Role: LEADER
I20260806 12:31:33.524828 21244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:33.525233 21239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:33.526022 21246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55283a3fa83f4e6abb774d221ab41256. Latest consensus state: current_term: 1 leader_uuid: "55283a3fa83f4e6abb774d221ab41256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } } }
I20260806 12:31:33.526161 21246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:33.526515 21245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55283a3fa83f4e6abb774d221ab41256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } } }
I20260806 12:31:33.526613 21245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:33.529155 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:33.514571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.6.152.190:37149)
0806 12:31:33.515027 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:33.515067 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:33.515115 (+ 48us) server_negotiation.cc:407] Connection header received
0806 12:31:33.515809 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:33.515826 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:33.516134 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:33.516393 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:33.517333 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:33.518305 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:33.519581 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:33.520164 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:33.520292 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:33.523100 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:33.523105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:33.523117 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:33.523192 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:33.525348 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:33.525903 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:33.525910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:33.525916 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:33.526073 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:33.526531 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:33.526537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:33.526541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:33.527100 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:33.527354 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:33.528122 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 12:31:33.528419 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":306,"thread_start_us":154,"threads_started":1}
W20260806 12:31:33.531502 21260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:31:33.531589 21260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:31:33.531713 21253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:33.532457 21253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:33.537633 21253 catalog_manager.cc:1347] Generated new cluster ID: c11e8c8b2ccb4401a78b303f77fc19f6
I20260806 12:31:33.537698 21253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:33.544109 21253 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:31:33.545231 21253 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:33.551149 21253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Generated new TSK 0
I20260806 12:31:33.551849 21253 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:31:33.585779 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37149
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:33.699862 21266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:33.700107 21266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:33.700150 21266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:33.704164 21266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:33.704216 21266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:33.704319 21266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:31:33.709380 21266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:33.710464 21266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:33.711514 21266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:33.720957 21275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:33.720945 21272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:33.721035 21271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:33.721110 21266 server_base.cc:1034] running on GCE node
I20260806 12:31:33.721612 21266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:33.722183 21266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:33.723356 21266 hybrid_clock.cc:648] HybridClock initialized: now 1786019493723335 us; error 32 us; skew 500 ppm
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:33.726235 21266 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:31:33.727267 21266 webserver.cc:460] Webserver started at http://127.6.152.129:38705/ using document root <none> and password file <none>
I20260806 12:31:33.727852 21266 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:33.727916 21266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:33.728155 21266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:33.730136 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "a84e7f020baa48b0a32178ad171e46ab"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "64d92076300383003e2445a99f49aa39"
server_key_iv: "e24da1e09808caf9cd3840a3bcd52a7c"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.730734 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "a84e7f020baa48b0a32178ad171e46ab"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "64d92076300383003e2445a99f49aa39"
server_key_iv: "e24da1e09808caf9cd3840a3bcd52a7c"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.734368 21266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 12:31:33.736724 21284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.737888 21266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:31:33.738031 21266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "a84e7f020baa48b0a32178ad171e46ab"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "64d92076300383003e2445a99f49aa39"
server_key_iv: "e24da1e09808caf9cd3840a3bcd52a7c"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.738140 21266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:33.750007 21266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:33.752724 21266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:33.752920 21266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:33.753469 21266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:33.754254 21266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:33.754325 21266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.754396 21266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:33.754449 21266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.763659 21266 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:39011
I20260806 12:31:33.763695 21404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:39011 every 8 connection(s)
I20260806 12:31:33.764542 21266 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:31:33.772573 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21266
I20260806 12:31:33.772733 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:31:33.773095 6754 external_mini_cluster.cc:1428] Setting key 4ef30a5c1a29a92a140e6f83b5638013
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:33.778200 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:33.766314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:60349 (local address 127.6.152.190:37149)
0806 12:31:33.766459 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:33.766463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:33.767526 (+ 1063us) server_negotiation.cc:407] Connection header received
0806 12:31:33.768332 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:33.768335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:33.768380 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:33.768455 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:33.769552 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:33.770083 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:33.771205 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:33.771431 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:33.771508 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:33.774393 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:33.774396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:33.774398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:33.774422 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:33.775936 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:33.776606 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:33.776609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:33.776610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:33.776694 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:33.777103 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:33.777106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:33.777107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:33.777283 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:33.777362 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:33.777977 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 12:31:33.778082 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:31:33.779515 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:33.766677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:60349)
0806 12:31:33.767374 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:33.767428 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:33.768162 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:33.768615 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:33.768622 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:33.768959 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:33.769429 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:33.769442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:33.770188 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:33.770191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:33.771060 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:33.771069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:33.771561 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:33.771563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:33.771701 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:33.772357 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:33.772384 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:33.774211 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:33.776067 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:33.776073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:33.776087 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:33.776488 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:33.776794 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:33.776800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:33.776802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:33.777005 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:33.777395 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:33.777403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:33.777723 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 12:31:33.777934 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 12:31:33.778379 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":109,"threads_started":1}
I20260806 12:31:33.780864 21405 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37149
I20260806 12:31:33.781113 21405 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:33.781714 21405 heartbeater.cc:511] Master 127.6.152.190:37149 requested a full tablet report, sending...
I20260806 12:31:33.783262 21200 ts_manager.cc:194] Registered new tserver with Master: a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011)
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:31:33.784442 21200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:60349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:31:33.800213 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37149
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:33.909269 21410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:33.909564 21410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:33.909642 21410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:33.913575 21410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:33.913669 21410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:33.913759 21410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:31:33.918938 21410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:33.919971 21410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:33.921095 21410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:33.931212 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:33.931314 21415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:33.931366 21410 server_base.cc:1034] running on GCE node
W20260806 12:31:33.931219 21416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:33.931967 21410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:33.932612 21410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:33.933807 21410 hybrid_clock.cc:648] HybridClock initialized: now 1786019493933797 us; error 31 us; skew 500 ppm
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:33.936676 21410 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:31:33.937513 21410 webserver.cc:460] Webserver started at http://127.6.152.130:38291/ using document root <none> and password file <none>
I20260806 12:31:33.938056 21410 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:33.938150 21410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:33.938380 21410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:33.940389 21410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "ef57eb4acf134530aae0d926611f6e44"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "96ae35b80d943cb0343f67c0f008f53a"
server_key_iv: "3b2581f6c84e836a7dac89af154131c1"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.941015 21410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "ef57eb4acf134530aae0d926611f6e44"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "96ae35b80d943cb0343f67c0f008f53a"
server_key_iv: "3b2581f6c84e836a7dac89af154131c1"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.944772 21410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 12:31:33.947165 21426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.948326 21410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:31:33.948480 21410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "ef57eb4acf134530aae0d926611f6e44"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "96ae35b80d943cb0343f67c0f008f53a"
server_key_iv: "3b2581f6c84e836a7dac89af154131c1"
server_key_version: "encryptionkey@0"
I20260806 12:31:33.948592 21410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:33.980754 21410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:33.983929 21410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:33.984140 21410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:33.984766 21410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:33.985667 21410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:33.985743 21410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.985819 21410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:33.985872 21410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:33.995266 21410 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:42187
I20260806 12:31:33.995321 21550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:42187 every 8 connection(s)
I20260806 12:31:33.996212 21410 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:31:33 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:34.006124 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21410
I20260806 12:31:34.006265 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:31:34.006589 6754 external_mini_cluster.cc:1428] Setting key bc841f9227be169a1e154deada22df10
I20260806 12:31:34.010016 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:33.998140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:54799 (local address 127.6.152.190:37149)
0806 12:31:33.998282 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:33.998285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:33.999355 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 12:31:34.000321 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.000325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.000387 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.000493 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:34.001695 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.002211 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.003431 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.003661 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.003776 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.006365 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:34.006370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:34.006373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:34.006416 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:34.007984 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.008489 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.008494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.008496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.008578 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.008961 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.008964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.008966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.009148 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:34.009259 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.009741 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.009858 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:31:34.010807 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:33.998496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:54799)
0806 12:31:33.999191 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:33.999247 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.000102 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.000675 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.000682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.001055 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:34.001543 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.001557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.002343 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.002348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.003286 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.003297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.003833 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.003837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.004010 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.004577 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:34.004593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:34.006125 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:34.008097 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.008103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.008117 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.008380 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.008683 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.008687 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.008689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.008841 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.009260 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:34.009268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:34.009510 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.009695 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.010019 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":105,"threads_started":1}
I20260806 12:31:34.012048 21551 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37149
I20260806 12:31:34.012307 21551 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:34.012959 21551 heartbeater.cc:511] Master 127.6.152.190:37149 requested a full tablet report, sending...
I20260806 12:31:34.014145 21200 ts_manager.cc:194] Registered new tserver with Master: ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187)
I20260806 12:31:34.015028 21200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:54799
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:31:34.035521 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37149
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:34.146154 21556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:34.146401 21556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:34.146445 21556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:34.150480 21556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:34.150540 21556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:34.150597 21556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:31:34.155887 21556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:34.156826 21556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:34.157763 21556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:34.167285 21562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:34.167311 21563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:34.167304 21556 server_base.cc:1034] running on GCE node
W20260806 12:31:34.167315 21565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:34.167975 21556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:34.168547 21556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:34.169704 21556 hybrid_clock.cc:648] HybridClock initialized: now 1786019494169675 us; error 42 us; skew 500 ppm
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:34.172508 21556 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:31:34.173359 21556 webserver.cc:460] Webserver started at http://127.6.152.131:39687/ using document root <none> and password file <none>
I20260806 12:31:34.173897 21556 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:34.173967 21556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:34.174213 21556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:34.176254 21556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "118e53bedd0140c6adf15aec70d548f1"
format_stamp: "Formatted at 2026-08-06 12:31:34 on dist-test-slave-1xrh"
server_key: "8b5e81284e998dd9f8d26849f34e86e1"
server_key_iv: "cdcb9edd002dec14cb2754f204c70f29"
server_key_version: "encryptionkey@0"
I20260806 12:31:34.176824 21556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "118e53bedd0140c6adf15aec70d548f1"
format_stamp: "Formatted at 2026-08-06 12:31:34 on dist-test-slave-1xrh"
server_key: "8b5e81284e998dd9f8d26849f34e86e1"
server_key_iv: "cdcb9edd002dec14cb2754f204c70f29"
server_key_version: "encryptionkey@0"
I20260806 12:31:34.180431 21556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:31:34.182900 21573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:34.184151 21556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:34.184249 21556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "118e53bedd0140c6adf15aec70d548f1"
format_stamp: "Formatted at 2026-08-06 12:31:34 on dist-test-slave-1xrh"
server_key: "8b5e81284e998dd9f8d26849f34e86e1"
server_key_iv: "cdcb9edd002dec14cb2754f204c70f29"
server_key_version: "encryptionkey@0"
I20260806 12:31:34.184335 21556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:34.200356 21556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:34.203011 21556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:34.203222 21556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:34.203735 21556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:34.204480 21556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:34.204527 21556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:34.204566 21556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:34.204591 21556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:34.213189 21556 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35363
I20260806 12:31:34.213217 21689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35363 every 8 connection(s)
I20260806 12:31:34.214092 21556 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:31:34.221406 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21556
I20260806 12:31:34.221560 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:31:34.221884 6754 external_mini_cluster.cc:1428] Setting key a174ab0264b3a7f3d2f84263d964accb
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:34.227434 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.215951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:53357 (local address 127.6.152.190:37149)
0806 12:31:34.216094 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.216098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.217046 (+ 948us) server_negotiation.cc:407] Connection header received
0806 12:31:34.217863 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.217866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.217912 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.217985 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:34.219122 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.219768 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.220848 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.221056 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.221160 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.223943 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:34.223947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:34.223949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:34.223972 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:34.225214 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.225823 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.225830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.225832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.225905 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.226244 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.226249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.226251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.226594 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:34.226718 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.227214 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.227308 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:31:34.228224 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.216249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:53357)
0806 12:31:34.216886 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.216929 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.217663 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.218149 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.218155 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.218507 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:34.218956 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.218969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.219881 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.219884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.220739 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.220748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.221181 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.221184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.221316 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.222061 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:34.222077 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:34.223755 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:34.225334 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.225340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.225354 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.225711 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.225999 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.226002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.226004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.226143 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.226742 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:34.226748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:34.226960 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.227116 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.227418 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":114,"threads_started":1}
I20260806 12:31:34.229276 21690 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37149
I20260806 12:31:34.229497 21690 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:34.229943 21690 heartbeater.cc:511] Master 127.6.152.190:37149 requested a full tablet report, sending...
I20260806 12:31:34.230965 21200 ts_manager.cc:194] Registered new tserver with Master: 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
I20260806 12:31:34.231631 21200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:53357
I20260806 12:31:34.235644 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:31:34.244441 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.237569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.6.152.190:37149)
0806 12:31:34.237678 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.237681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.237896 (+ 215us) server_negotiation.cc:407] Connection header received
0806 12:31:34.237989 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.237991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.238024 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.238087 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:34.238706 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.239240 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.240007 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.240215 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.240286 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.241682 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:34.241690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:34.241692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:34.241712 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:34.242961 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.243442 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.243445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.243446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.243516 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.243755 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.243757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.243758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.243962 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:34.244065 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.244242 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.244324 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:31:34.247294 21200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:31:34.250254 21200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:31:34.264818 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.259063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.6.152.131:35363)
0806 12:31:34.259990 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.260006 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.260030 (+ 24us) server_negotiation.cc:407] Connection header received
0806 12:31:34.260108 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.260112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.260330 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.260789 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:34.260905 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.261631 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.263208 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.264071 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.264361 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.264526 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.264579 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.264665 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":182,"threads_started":1}
I20260806 12:31:34.264807 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.258373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42187 (local address 127.0.0.1:51212)
0806 12:31:34.258806 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.258857 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.258977 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.259740 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.259748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.259775 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:34.259957 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.259966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.261644 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.261650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.263081 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.263092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.264224 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.264228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.264366 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.264389 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.264456 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.264618 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":68,"thread_start_us":80,"threads_started":1}
I20260806 12:31:34.264813 21701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.258465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35363 (local address 127.0.0.1:49706)
0806 12:31:34.259206 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.259219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.259294 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.260490 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.260493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.260516 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:34.260643 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.260649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.261797 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.261801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.263043 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.263054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.264224 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.264228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.264392 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.264413 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.264470 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.264638 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":158,"spinlock_wait_cycles":2816,"thread_start_us":97,"threads_started":1}
I20260806 12:31:34.264813 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.259287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39011 (local address 127.0.0.1:32860)
0806 12:31:34.259568 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.259581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.259667 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.260461 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.260467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.260490 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:34.260643 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.260649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.262003 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.262007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.263291 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.263299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.264150 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.264154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.264300 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.264356 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.264469 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.264636 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":140,"thread_start_us":80,"threads_started":1}
I20260806 12:31:34.265587 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.258477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.6.152.130:42187)
0806 12:31:34.259236 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.259251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.259277 (+ 26us) server_negotiation.cc:407] Connection header received
0806 12:31:34.259358 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.259366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.259572 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.260127 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:34.260322 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.261477 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.263249 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.264071 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.265137 (+ 1066us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.265284 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.265329 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.265426 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":126,"threads_started":1}
I20260806 12:31:34.266153 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.259634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.6.152.129:39011)
0806 12:31:34.259981 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.259997 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.260022 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:31:34.260097 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.260101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.260306 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.260469 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:34.260769 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.261857 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.263419 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.264000 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.264137 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.264296 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.264884 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.266006 (+ 1122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":95,"threads_started":1}
I20260806 12:31:34.267695 21623 tablet_service.cc:1467] Processing CreateTablet for tablet 3de2f29e1c5a4dc29ae5442c7d65130c (DEFAULT_TABLE table=test-table [id=0641254cac114a03ba4077836f7c6119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:34.267948 21480 tablet_service.cc:1467] Processing CreateTablet for tablet 3de2f29e1c5a4dc29ae5442c7d65130c (DEFAULT_TABLE table=test-table [id=0641254cac114a03ba4077836f7c6119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:34.268716 21623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3de2f29e1c5a4dc29ae5442c7d65130c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:34.268931 21480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3de2f29e1c5a4dc29ae5442c7d65130c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:34.270674 21333 tablet_service.cc:1467] Processing CreateTablet for tablet 3de2f29e1c5a4dc29ae5442c7d65130c (DEFAULT_TABLE table=test-table [id=0641254cac114a03ba4077836f7c6119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:34.271627 21333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3de2f29e1c5a4dc29ae5442c7d65130c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:34.275344 21707 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Bootstrap starting.
I20260806 12:31:34.276244 21708 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Bootstrap starting.
I20260806 12:31:34.277503 21707 tablet_bootstrap.cc:654] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:34.278419 21709 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Bootstrap starting.
I20260806 12:31:34.278793 21708 tablet_bootstrap.cc:654] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:34.278769 21707 log.cc:824] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:34.279690 21708 log.cc:824] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:34.280911 21709 tablet_bootstrap.cc:654] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:34.281162 21707 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: No bootstrap required, opened a new log
I20260806 12:31:34.281410 21707 ts_tablet_manager.cc:1397] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260806 12:31:34.281689 21708 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: No bootstrap required, opened a new log
I20260806 12:31:34.281759 21709 log.cc:824] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:34.281884 21708 ts_tablet_manager.cc:1397] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:31:34.283578 21709 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: No bootstrap required, opened a new log
I20260806 12:31:34.283758 21709 ts_tablet_manager.cc:1397] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:31:34.284701 21707 raft_consensus.cc:348] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.284952 21707 raft_consensus.cc:374] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:34.285008 21707 raft_consensus.cc:729] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118e53bedd0140c6adf15aec70d548f1, State: Initialized, Role: FOLLOWER
I20260806 12:31:34.285446 21707 consensus_queue.cc:260] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.285742 21708 raft_consensus.cc:348] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.285996 21708 raft_consensus.cc:374] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:34.286052 21708 raft_consensus.cc:729] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57eb4acf134530aae0d926611f6e44, State: Initialized, Role: FOLLOWER
I20260806 12:31:34.286135 21690 heartbeater.cc:503] Master 127.6.152.190:37149 was elected leader, sending a full tablet report...
I20260806 12:31:34.286433 21707 ts_tablet_manager.cc:1428] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 12:31:34.286460 21708 consensus_queue.cc:260] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.287185 21551 heartbeater.cc:503] Master 127.6.152.190:37149 was elected leader, sending a full tablet report...
I20260806 12:31:34.287434 21708 ts_tablet_manager.cc:1428] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 12:31:34.288621 21709 raft_consensus.cc:348] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.288872 21709 raft_consensus.cc:374] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:34.288931 21709 raft_consensus.cc:729] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84e7f020baa48b0a32178ad171e46ab, State: Initialized, Role: FOLLOWER
I20260806 12:31:34.289331 21709 consensus_queue.cc:260] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.290110 21709 ts_tablet_manager.cc:1428] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:31:34.290236 21405 heartbeater.cc:503] Master 127.6.152.190:37149 was elected leader, sending a full tablet report...
I20260806 12:31:34.337150 21713 raft_consensus.cc:482] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:34.337407 21713 raft_consensus.cc:504] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.338553 21713 leader_election.cc:288] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011)
I20260806 12:31:34.343166 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.338818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42187 (local address 127.6.152.131:33259)
0806 12:31:34.339077 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.339094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.339217 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.339493 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.339496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.339520 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:34.339658 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.339668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.340725 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.340727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.341929 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.341941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.342682 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.342686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.342797 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.342834 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.342921 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.343008 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260806 12:31:34.343220 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.339081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:33259 (local address 127.6.152.130:42187)
0806 12:31:34.339233 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.339237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.339255 (+ 18us) server_negotiation.cc:407] Connection header received
0806 12:31:34.339315 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.339321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.339374 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.339464 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:34.339844 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.340614 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.342074 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.342568 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.342668 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.342710 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.342949 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.343034 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 12:31:34.343441 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.339111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39011 (local address 127.6.152.131:33495)
0806 12:31:34.339602 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.339619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.339722 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.339982 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.339984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.339999 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:34.340092 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.340096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.341050 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.341054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.342156 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.342163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.342945 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.342948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.343057 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.343080 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.343184 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.343265 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":1,"thread_start_us":135,"threads_started":1}
I20260806 12:31:34.343477 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.339279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:33495 (local address 127.6.152.129:39011)
0806 12:31:34.339419 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.339426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.339705 (+ 279us) server_negotiation.cc:407] Connection header received
0806 12:31:34.339827 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.339830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.339883 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.339968 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:34.340209 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.340935 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.342267 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.342796 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.342926 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.342972 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.343221 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.343311 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:31:34.343851 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de2f29e1c5a4dc29ae5442c7d65130c" candidate_uuid: "118e53bedd0140c6adf15aec70d548f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57eb4acf134530aae0d926611f6e44" is_pre_election: true
I20260806 12:31:34.344187 21504 raft_consensus.cc:2454] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 118e53bedd0140c6adf15aec70d548f1 in term 0.
I20260806 12:31:34.344380 21355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de2f29e1c5a4dc29ae5442c7d65130c" candidate_uuid: "118e53bedd0140c6adf15aec70d548f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84e7f020baa48b0a32178ad171e46ab" is_pre_election: true
I20260806 12:31:34.344650 21355 raft_consensus.cc:2454] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 118e53bedd0140c6adf15aec70d548f1 in term 0.
I20260806 12:31:34.344687 21575 leader_election.cc:302] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118e53bedd0140c6adf15aec70d548f1, ef57eb4acf134530aae0d926611f6e44; no voters:
I20260806 12:31:34.344949 21713 raft_consensus.cc:2781] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:31:34.345036 21713 raft_consensus.cc:482] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:31:34.345079 21713 raft_consensus.cc:3037] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:34.346171 21713 raft_consensus.cc:504] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.346503 21713 leader_election.cc:288] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [CANDIDATE]: Term 1 election: Requested vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011)
I20260806 12:31:34.346843 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de2f29e1c5a4dc29ae5442c7d65130c" candidate_uuid: "118e53bedd0140c6adf15aec70d548f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57eb4acf134530aae0d926611f6e44"
I20260806 12:31:34.346882 21355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de2f29e1c5a4dc29ae5442c7d65130c" candidate_uuid: "118e53bedd0140c6adf15aec70d548f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84e7f020baa48b0a32178ad171e46ab"
I20260806 12:31:34.346943 21504 raft_consensus.cc:3037] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:34.347030 21355 raft_consensus.cc:3037] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:34.348096 21504 raft_consensus.cc:2454] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 118e53bedd0140c6adf15aec70d548f1 in term 1.
I20260806 12:31:34.348124 21355 raft_consensus.cc:2454] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 118e53bedd0140c6adf15aec70d548f1 in term 1.
I20260806 12:31:34.348419 21575 leader_election.cc:302] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118e53bedd0140c6adf15aec70d548f1, a84e7f020baa48b0a32178ad171e46ab; no voters:
I20260806 12:31:34.348609 21713 raft_consensus.cc:2781] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:34.348826 21713 raft_consensus.cc:686] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 1 LEADER]: Becoming Leader. State: Replica: 118e53bedd0140c6adf15aec70d548f1, State: Running, Role: LEADER
I20260806 12:31:34.349151 21713 consensus_queue.cc:237] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.352216 21199 catalog_manager.cc:5495] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "118e53bedd0140c6adf15aec70d548f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } health_report { overall_health: UNKNOWN } } }
I20260806 12:31:34.385377 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.381876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.6.152.131:35363)
0806 12:31:34.382013 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.382018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.382057 (+ 39us) server_negotiation.cc:407] Connection header received
0806 12:31:34.382282 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.382285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.382345 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.382435 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:31:34.382653 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.383163 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.383988 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.384212 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.384294 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.384564 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:31:34.384969 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:31:34.385070 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.385117 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.385166 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:31:34.391484 21355 raft_consensus.cc:1261] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Refusing update from remote peer 118e53bedd0140c6adf15aec70d548f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:34.391484 21504 raft_consensus.cc:1261] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Refusing update from remote peer 118e53bedd0140c6adf15aec70d548f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:34.392261 21717 consensus_queue.cc:1035] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:34.392467 21713 consensus_queue.cc:1035] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:34.400763 21721 mvcc.cc:205] Tried to move back new op lower bound from 7315535849019650048 to 7315535848856608768. Current Snapshot: MvccSnapshot[applied={T|T < 7315535849019650048}]
I20260806 12:31:34.404183 21722 mvcc.cc:205] Tried to move back new op lower bound from 7315535849019650048 to 7315535848856608768. Current Snapshot: MvccSnapshot[applied={T|T < 7315535849019650048 or (T in {7315535849019650048})}]
I20260806 12:31:34.409050 21199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:34.409221 21199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:34.411494 21199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Sending DeleteTablet for 3 replicas of tablet 3de2f29e1c5a4dc29ae5442c7d65130c
I20260806 12:31:34.412369 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21266
I20260806 12:31:34.412544 21480 tablet_service.cc:1514] Processing DeleteTablet for tablet 3de2f29e1c5a4dc29ae5442c7d65130c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:34 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:51212
I20260806 12:31:34.413025 21623 tablet_service.cc:1514] Processing DeleteTablet for tablet 3de2f29e1c5a4dc29ae5442c7d65130c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:34 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:49706
I20260806 12:31:34.414577 21730 tablet_replica.cc:330] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: stopping tablet replica
I20260806 12:31:34.414950 21730 raft_consensus.cc:2229] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:31:34.415324 21730 raft_consensus.cc:2258] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:34.416992 21730 ts_tablet_manager.cc:1905] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:31:34.420643 21730 ts_tablet_manager.cc:1918] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:34.420756 21730 log.cc:1196] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/3de2f29e1c5a4dc29ae5442c7d65130c
I20260806 12:31:34.421032 21731 tablet_replica.cc:330] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: stopping tablet replica
I20260806 12:31:34.421140 21730 ts_tablet_manager.cc:1939] T 3de2f29e1c5a4dc29ae5442c7d65130c P ef57eb4acf134530aae0d926611f6e44: Deleting consensus metadata
I20260806 12:31:34.421437 21731 raft_consensus.cc:2229] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:31:34.421864 21731 raft_consensus.cc:2258] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 12:31:34.422143 21184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:39011 (error 108)
I20260806 12:31:34.422646 21184 catalog_manager.cc:4841] TS ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): tablet 3de2f29e1c5a4dc29ae5442c7d65130c (table test-table [id=0641254cac114a03ba4077836f7c6119]) successfully deleted
W20260806 12:31:34.423329 21705 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:31:34.422959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39011 (local address 127.0.0.1:32874)
0806 12:31:34.423098 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.423116 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.423214 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39011: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 12:31:34.423532 21731 ts_tablet_manager.cc:1905] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 12:31:34.423511 21184 catalog_manager.cc:4584] TS a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3de2f29e1c5a4dc29ae5442c7d65130c: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39011: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 12:31:34.423550 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21410
I20260806 12:31:34.426823 21731 ts_tablet_manager.cc:1918] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:34.426967 21731 log.cc:1196] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/3de2f29e1c5a4dc29ae5442c7d65130c
I20260806 12:31:34.427398 21731 ts_tablet_manager.cc:1939] T 3de2f29e1c5a4dc29ae5442c7d65130c P 118e53bedd0140c6adf15aec70d548f1: Deleting consensus metadata
I20260806 12:31:34.428493 21186 catalog_manager.cc:4841] TS 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): tablet 3de2f29e1c5a4dc29ae5442c7d65130c (table test-table [id=0641254cac114a03ba4077836f7c6119]) successfully deleted
I20260806 12:31:34.437270 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21556
I20260806 12:31:34.444092 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21165
I20260806 12:31:34.450395 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37149
--webserver_interface=127.6.152.190
--webserver_port=46459
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:34.559101 21733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:34.559404 21733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:34.559484 21733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:34.563510 21733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:34.563593 21733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:34.563633 21733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:34.563692 21733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:34.563735 21733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:34.568774 21733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:34.569751 21733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:34.570731 21733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:34.584317 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:34.584384 21733 server_base.cc:1034] running on GCE node
W20260806 12:31:34.584317 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:34.584594 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:34.585232 21733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:34.586342 21733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:34.587503 21733 hybrid_clock.cc:648] HybridClock initialized: now 1786019494587488 us; error 26 us; skew 500 ppm
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:34.590294 21733 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:34.591218 21733 webserver.cc:460] Webserver started at http://127.6.152.190:46459/ using document root <none> and password file <none>
I20260806 12:31:34.591800 21733 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:34.591914 21733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:34.595752 21733 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 12:31:34.597868 21751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:34.599080 21733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:34.599252 21733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "55283a3fa83f4e6abb774d221ab41256"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "f1be9789f59dfb4ec9420849e0f1145a"
server_key_iv: "4a253202befd15525e112b3cd3006178"
server_key_version: "encryptionkey@0"
I20260806 12:31:34.599877 21733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:34.614936 21733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:34.617743 21733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:34.617944 21733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:34.625013 21733 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37149
I20260806 12:31:34.625301 21806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37149 every 8 connection(s)
I20260806 12:31:34.626087 21733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:34.626711 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21733
I20260806 12:31:34.627256 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:39011
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:37149
--webserver_port=38705
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:34.630640 21807 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:31:34.632428 21807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap starting.
I20260806 12:31:34.650354 21807 log.cc:824] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:34.658671 21807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:31:34.659050 21807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap complete.
I20260806 12:31:34.663170 21807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:34.663708 21807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55283a3fa83f4e6abb774d221ab41256, State: Initialized, Role: FOLLOWER
I20260806 12:31:34.664215 21807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:34.664358 21807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:34.664460 21807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:34.664584 21807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:31:34.666710 21807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:34.667055 21807 leader_election.cc:302] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55283a3fa83f4e6abb774d221ab41256; no voters:
I20260806 12:31:34.667441 21807 leader_election.cc:288] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:31:34.667548 21811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:31:34.667778 21811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 LEADER]: Becoming Leader. State: Replica: 55283a3fa83f4e6abb774d221ab41256, State: Running, Role: LEADER
I20260806 12:31:34.668083 21811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:34.668727 21807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:34.669035 21811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "55283a3fa83f4e6abb774d221ab41256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } } }
I20260806 12:31:34.669138 21811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:34.669332 21811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55283a3fa83f4e6abb774d221ab41256. Latest consensus state: current_term: 2 leader_uuid: "55283a3fa83f4e6abb774d221ab41256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } } }
I20260806 12:31:34.669435 21811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:34.670524 21815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:34.675542 21815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:34.676234 21815 catalog_manager.cc:1259] Loaded cluster ID: c11e8c8b2ccb4401a78b303f77fc19f6
I20260806 12:31:34.676340 21815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:34.677889 21815 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 12:31:34.682895 21815 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 12:31:34.688297 21815 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:34.689004 21815 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Loaded TSK: 0
I20260806 12:31:34.690166 21815 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:31:34.740882 21809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:34.741151 21809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:34.741232 21809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:34.745349 21809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:34.745431 21809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:34.745519 21809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:31:34.750607 21809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:39011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=38705
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:34.751639 21809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:34.752596 21809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:34.763053 21835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:34.763106 21837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:34.763051 21834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:34.763499 21809 server_base.cc:1034] running on GCE node
I20260806 12:31:34.764067 21809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:34.764806 21809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:34.766005 21809 hybrid_clock.cc:648] HybridClock initialized: now 1786019494765981 us; error 31 us; skew 500 ppm
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:34.769362 21809 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:31:34.770468 21809 webserver.cc:460] Webserver started at http://127.6.152.129:38705/ using document root <none> and password file <none>
I20260806 12:31:34.771158 21809 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:34.771273 21809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:34.775204 21809 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:31:34.777379 21844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:34.778640 21809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:34.778765 21809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "a84e7f020baa48b0a32178ad171e46ab"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "64d92076300383003e2445a99f49aa39"
server_key_iv: "e24da1e09808caf9cd3840a3bcd52a7c"
server_key_version: "encryptionkey@0"
I20260806 12:31:34.779335 21809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:34.790215 21809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:34.793452 21809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:34.793659 21809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:34.794219 21809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:34.795420 21851 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:31:34.799378 21809 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:31:34.799446 21809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260806 12:31:34.799513 21809 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:31:34.801126 21809 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:31:34.801184 21809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 12:31:34.801429 21851 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Bootstrap starting.
I20260806 12:31:34.812398 21809 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:39011
I20260806 12:31:34.812670 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:39011 every 8 connection(s)
I20260806 12:31:34.813575 21809 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:31:34.814384 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21809
I20260806 12:31:34.814857 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:42187
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:37149
--webserver_port=38291
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:34.825868 21851 log.cc:824] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:34.834599 21851 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:31:34.835009 21851 tablet_bootstrap.cc:492] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Bootstrap complete.
I20260806 12:31:34.835500 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.816207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:43495)
0806 12:31:34.817455 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.817516 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.818562 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.820187 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.820199 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.820760 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:34.821452 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.821478 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.823606 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.823611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.825206 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.825220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.826375 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.826379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.826572 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.827301 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:34.827328 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:34.829545 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:34.832071 (+ 2526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.832077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.832090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.832433 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.832835 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.832838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.832840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.833005 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.833698 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:34.833704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:34.833966 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.834172 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.834487 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"thread_start_us":149,"threads_started":1}
I20260806 12:31:34.835549 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.816166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:43495 (local address 127.6.152.190:37149)
0806 12:31:34.818369 (+ 2203us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.818386 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.818442 (+ 56us) server_negotiation.cc:407] Connection header received
0806 12:31:34.818967 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.818983 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.819387 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.819727 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:34.821677 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.823396 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.825369 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.826180 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.826367 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.829756 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:34.829762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:34.829778 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:34.829811 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:34.831934 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.832563 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.832569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.832574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.832717 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.833117 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.833120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.833122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.833554 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:34.833674 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.834296 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.834583 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2062,"thread_start_us":123,"threads_started":1}
I20260806 12:31:34.836562 21851 ts_tablet_manager.cc:1397] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Time spent bootstrapping tablet: real 0.035s user 0.022s sys 0.011s
I20260806 12:31:34.838423 21961 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37149
I20260806 12:31:34.838656 21961 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:34.839211 21961 heartbeater.cc:511] Master 127.6.152.190:37149 requested a full tablet report, sending...
I20260806 12:31:34.839466 21851 raft_consensus.cc:348] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.839880 21851 raft_consensus.cc:729] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84e7f020baa48b0a32178ad171e46ab, State: Initialized, Role: FOLLOWER
I20260806 12:31:34.840293 21851 consensus_queue.cc:260] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:34.841338 21851 ts_tablet_manager.cc:1428] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:31:34.843101 21770 ts_manager.cc:194] Registered new tserver with Master: a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011)
I20260806 12:31:34.846477 21770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:43495
I20260806 12:31:34.848974 21961 heartbeater.cc:503] Master 127.6.152.190:37149 was elected leader, sending a full tablet report...
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:31:34.856369 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.845805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39011 (local address 127.0.0.1:32890)
0806 12:31:34.846397 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:34.846457 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:34.846632 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:34.847179 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:34.847185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:34.847610 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:34.847734 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.847742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.849140 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.849144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.849926 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:34.849936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.850621 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.850624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:34.850728 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.851349 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:34.851362 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:34.852684 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:34.854592 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.854604 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.854609 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.854838 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.855213 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:34.855216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:34.855218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:34.855306 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:34.855757 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:34.855763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:34.855883 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 12:31:34.856020 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 12:31:34.856202 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":107,"threads_started":1}
I20260806 12:31:34.856401 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:34.845900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.6.152.129:39011)
0806 12:31:34.846505 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:34.846517 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:34.846560 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:31:34.846799 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:34.846806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:34.846993 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:34.847168 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:34.847870 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.848967 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.850058 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:34.850432 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:34.850546 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:34.852832 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:34.852841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:34.852850 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:34.852871 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:34.854466 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.854981 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.854986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.854987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.855069 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:34.855416 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:34.855419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:34.855420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:34.855641 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:34.855779 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:34.856109 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 12:31:34.856263 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":125,"threads_started":1}
I20260806 12:31:34.857250 21893 tablet_service.cc:1514] Processing DeleteTablet for tablet 3de2f29e1c5a4dc29ae5442c7d65130c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:34 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:32890
I20260806 12:31:34.858090 21970 tablet_replica.cc:330] stopping tablet replica
I20260806 12:31:34.858392 21970 raft_consensus.cc:2229] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:31:34.858639 21970 raft_consensus.cc:2258] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:34.859874 21970 ts_tablet_manager.cc:1905] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:31:34.864291 21970 ts_tablet_manager.cc:1918] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:34.864414 21970 log.cc:1196] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/3de2f29e1c5a4dc29ae5442c7d65130c
I20260806 12:31:34.864707 21970 ts_tablet_manager.cc:1939] T 3de2f29e1c5a4dc29ae5442c7d65130c P a84e7f020baa48b0a32178ad171e46ab: Deleting consensus metadata
I20260806 12:31:34.866410 21754 catalog_manager.cc:4841] TS a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011): tablet 3de2f29e1c5a4dc29ae5442c7d65130c (table test-table [id=0641254cac114a03ba4077836f7c6119]) successfully deleted
W20260806 12:31:34.937544 21965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:34.937793 21965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:34.937868 21965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:34.941777 21965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:34.941828 21965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:34.941926 21965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:31:34.947017 21965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:42187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=38291
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:34.948055 21965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:34.949046 21965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:34.958247 21975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:34.958225 21976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:34.958469 21978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:34.958448 21965 server_base.cc:1034] running on GCE node
I20260806 12:31:34.958845 21965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:34.959540 21965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:34.960726 21965 hybrid_clock.cc:648] HybridClock initialized: now 1786019494960701 us; error 44 us; skew 500 ppm
Aug 06 12:31:34 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:34.963384 21965 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:31:34.964241 21965 webserver.cc:460] Webserver started at http://127.6.152.130:38291/ using document root <none> and password file <none>
I20260806 12:31:34.964772 21965 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:34.964879 21965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:34.968514 21965 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:34.970579 21985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:34.971855 21965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:34.971952 21965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "ef57eb4acf134530aae0d926611f6e44"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "96ae35b80d943cb0343f67c0f008f53a"
server_key_iv: "3b2581f6c84e836a7dac89af154131c1"
server_key_version: "encryptionkey@0"
I20260806 12:31:34.972451 21965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:34.996609 21965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:34.999492 21965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:34.999639 21965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:35.000160 21965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:35.000934 21965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:35.000982 21965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.001022 21965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:35.001044 21965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.010113 21965 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:42187
I20260806 12:31:35.010149 22100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:42187 every 8 connection(s)
I20260806 12:31:35.011118 21965 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:31:35.013504 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 21965
I20260806 12:31:35.014042 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:35363
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:37149
--webserver_port=39687
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:31:35 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:35.026747 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.012995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:37225 (local address 127.6.152.190:37149)
0806 12:31:35.013139 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.013143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.014395 (+ 1252us) server_negotiation.cc:407] Connection header received
0806 12:31:35.015817 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.015823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.015901 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.016017 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:35.017247 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.017775 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.019711 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.019945 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.020029 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.023004 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:35.023008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:35.023010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:35.023035 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:35.024634 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:35.025178 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:35.025182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:35.025184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:35.025267 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:35.025641 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:35.025644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:35.025646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:35.025832 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:35.025950 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.026422 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.026562 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:31:35.027567 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.013367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:37225)
0806 12:31:35.014223 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.014282 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.015563 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.016199 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:35.016206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:35.016557 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:35.017064 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.017088 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.017920 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.017926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.019579 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.019593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.020120 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.020126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.020360 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.021007 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:35.021043 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:35.022806 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:35.024766 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:35.024772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:35.024786 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:35.025060 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:35.025363 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:35.025367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:35.025369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:35.025529 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:35.025950 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:35.025957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:35.026188 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 12:31:35.026377 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 12:31:35.026708 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":151,"threads_started":1}
I20260806 12:31:35.028820 22101 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37149
I20260806 12:31:35.029089 22101 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:35.029613 22101 heartbeater.cc:511] Master 127.6.152.190:37149 requested a full tablet report, sending...
I20260806 12:31:35.030669 21770 ts_manager.cc:194] Registered new tserver with Master: ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187)
I20260806 12:31:35.031304 21770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:37225
W20260806 12:31:35.128500 22104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:35.128736 22104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:35.128818 22104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:35.132975 22104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:35.133025 22104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:35.133128 22104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:31:35.138432 22104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:35363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=39687
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:35.139492 22104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:35.140479 22104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:35.149899 22110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:35.149892 22109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:35.149976 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:35.150141 22104 server_base.cc:1034] running on GCE node
I20260806 12:31:35.150584 22104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:35.151281 22104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:35.152467 22104 hybrid_clock.cc:648] HybridClock initialized: now 1786019495152455 us; error 32 us; skew 500 ppm
Aug 06 12:31:35 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:35.155282 22104 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:31:35.156086 22104 webserver.cc:460] Webserver started at http://127.6.152.131:39687/ using document root <none> and password file <none>
I20260806 12:31:35.156625 22104 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:35.156720 22104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:35.160503 22104 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 12:31:35.162443 22119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.163645 22104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:35.163780 22104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "118e53bedd0140c6adf15aec70d548f1"
format_stamp: "Formatted at 2026-08-06 12:31:34 on dist-test-slave-1xrh"
server_key: "8b5e81284e998dd9f8d26849f34e86e1"
server_key_iv: "cdcb9edd002dec14cb2754f204c70f29"
server_key_version: "encryptionkey@0"
I20260806 12:31:35.164301 22104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:35.192160 22104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:35.194938 22104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:35.195155 22104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:35.195760 22104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:35.196599 22104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:35.196645 22104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.196717 22104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:35.196758 22104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.206211 22104 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35363
I20260806 12:31:35.206244 22235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35363 every 8 connection(s)
I20260806 12:31:35.207288 22104 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:31:35.211834 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22104
Aug 06 12:31:35 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:35.221920 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.212527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.6.152.190:37149)
0806 12:31:35.212791 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.212794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.213022 (+ 228us) server_negotiation.cc:407] Connection header received
0806 12:31:35.213142 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.213144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.213180 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.213252 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:35.214164 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.214924 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.215987 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.216215 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.216323 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.218429 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:35.218433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:35.218436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:35.218475 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:35.220116 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:35.220597 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:35.220600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:35.220601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:35.220712 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:35.221021 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:35.221023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:35.221025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:35.221200 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:35.221445 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.221572 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.221724 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260806 12:31:35.222295 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.209094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:44111 (local address 127.6.152.190:37149)
0806 12:31:35.209237 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.209241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.210276 (+ 1035us) server_negotiation.cc:407] Connection header received
0806 12:31:35.211241 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.211244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.211305 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.211416 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:35.212702 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.213495 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.215017 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.215387 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.215505 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.217976 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:35.217983 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:35.217986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:35.218017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:35.219826 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:35.220450 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:35.220454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:35.220455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:35.220559 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:35.220973 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:35.220980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:35.220982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:35.221174 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:35.221250 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.222032 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.222157 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:31:35.223313 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.209465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:44111)
0806 12:31:35.210131 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.210171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.210990 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.211609 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:35.211616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:35.212018 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:35.212529 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.212548 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.213624 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.213627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.214853 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.214864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.215538 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.215541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.215684 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.216286 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:35.216302 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:35.217797 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:35.219947 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:35.219972 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:35.219989 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:35.220347 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:35.220655 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:35.220660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:35.220664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:35.220835 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:35.221290 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:35.221297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:35.221619 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 12:31:35.221932 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 12:31:35.222279 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":111,"threads_started":1}
I20260806 12:31:35.224310 22236 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37149
I20260806 12:31:35.224517 22236 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:35.224932 22236 heartbeater.cc:511] Master 127.6.152.190:37149 requested a full tablet report, sending...
I20260806 12:31:35.225912 21770 ts_manager.cc:194] Registered new tserver with Master: 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
I20260806 12:31:35.226500 21770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:44111
I20260806 12:31:35.235533 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:31:35.239451 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.236141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.6.152.190:37149)
0806 12:31:35.236252 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.236257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.236465 (+ 208us) server_negotiation.cc:407] Connection header received
0806 12:31:35.236574 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.236576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.236660 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.236739 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:31:35.236928 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.237412 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.238216 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.238420 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.238492 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.238761 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:31:35.239088 (+ 327us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:31:35.239275 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.239308 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.239350 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:31:35.240039 21770 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:31:35.243232 21770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:31:35.251252 21893 tablet_service.cc:1467] Processing CreateTablet for tablet e7047e9958844a76817eb1299cd50f1e (DEFAULT_TABLE table=test-table [id=cb58a6d3dba34cd1b660c06e8d5d0714]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:35.251868 21893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7047e9958844a76817eb1299cd50f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:35.254981 21851 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Bootstrap starting.
I20260806 12:31:35.255980 21851 tablet_bootstrap.cc:654] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:35.256125 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.250469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35363 (local address 127.0.0.1:49716)
0806 12:31:35.251262 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.251278 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.251368 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.252085 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:35.252088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:35.252108 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:35.252198 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.252203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.253729 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.253732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.254537 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.254547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.255639 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.255643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.255727 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.255778 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 12:31:35.255867 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 12:31:35.255964 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":3,"thread_start_us":118,"threads_started":1}
I20260806 12:31:35.256179 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.251207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.6.152.131:35363)
0806 12:31:35.251623 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.251636 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.251658 (+ 22us) server_negotiation.cc:407] Connection header received
0806 12:31:35.251728 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.251733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.251913 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.252079 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:35.252328 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.253594 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.254667 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.255493 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.255598 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.255736 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.255916 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.256029 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":100,"threads_started":1}
I20260806 12:31:35.256474 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.250266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42187 (local address 127.0.0.1:51226)
0806 12:31:35.250770 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.250786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.250881 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.251427 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:35.251430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:35.251451 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:35.251557 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.251564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.253519 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.253523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.254763 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.254774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.256106 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.256110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.256213 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.256240 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 12:31:35.256298 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:31:35.256385 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":25}
I20260806 12:31:35.256990 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.250453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.6.152.130:42187)
0806 12:31:35.250769 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.250781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.250902 (+ 121us) server_negotiation.cc:407] Connection header received
0806 12:31:35.250992 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.250996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.251238 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.251728 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:35.251890 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.253375 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.254918 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.255964 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.256618 (+ 654us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.256723 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.256763 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.256827 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":117,"threads_started":1}
I20260806 12:31:35.257682 21851 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: No bootstrap required, opened a new log
I20260806 12:31:35.257771 21851 ts_tablet_manager.cc:1397] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 12:31:35.258239 21851 raft_consensus.cc:348] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.258328 21851 raft_consensus.cc:374] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:35.258352 21851 raft_consensus.cc:729] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84e7f020baa48b0a32178ad171e46ab, State: Initialized, Role: FOLLOWER
I20260806 12:31:35.258482 21851 consensus_queue.cc:260] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.258746 21851 ts_tablet_manager.cc:1428] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:31:35.259948 22168 tablet_service.cc:1467] Processing CreateTablet for tablet e7047e9958844a76817eb1299cd50f1e (DEFAULT_TABLE table=test-table [id=cb58a6d3dba34cd1b660c06e8d5d0714]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:35.259944 22033 tablet_service.cc:1467] Processing CreateTablet for tablet e7047e9958844a76817eb1299cd50f1e (DEFAULT_TABLE table=test-table [id=cb58a6d3dba34cd1b660c06e8d5d0714]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:35.261030 22033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7047e9958844a76817eb1299cd50f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:35.261034 22168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7047e9958844a76817eb1299cd50f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:35.269357 22250 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Bootstrap starting.
I20260806 12:31:35.271492 22249 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Bootstrap starting.
I20260806 12:31:35.272009 22250 tablet_bootstrap.cc:654] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:35.272749 22250 log.cc:824] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:35.273411 22249 tablet_bootstrap.cc:654] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:35.274075 22249 log.cc:824] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:35.274675 22250 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: No bootstrap required, opened a new log
I20260806 12:31:35.274885 22250 ts_tablet_manager.cc:1397] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:31:35.275990 22249 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: No bootstrap required, opened a new log
I20260806 12:31:35.276202 22249 ts_tablet_manager.cc:1397] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 12:31:35.278162 22250 raft_consensus.cc:348] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.278383 22250 raft_consensus.cc:374] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:35.278450 22250 raft_consensus.cc:729] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118e53bedd0140c6adf15aec70d548f1, State: Initialized, Role: FOLLOWER
I20260806 12:31:35.278892 22250 consensus_queue.cc:260] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.279577 22236 heartbeater.cc:503] Master 127.6.152.190:37149 was elected leader, sending a full tablet report...
I20260806 12:31:35.279675 22249 raft_consensus.cc:348] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.279865 22250 ts_tablet_manager.cc:1428] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260806 12:31:35.279920 22249 raft_consensus.cc:374] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:35.279983 22249 raft_consensus.cc:729] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57eb4acf134530aae0d926611f6e44, State: Initialized, Role: FOLLOWER
I20260806 12:31:35.280375 22249 consensus_queue.cc:260] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.281019 22101 heartbeater.cc:503] Master 127.6.152.190:37149 was elected leader, sending a full tablet report...
I20260806 12:31:35.281277 22249 ts_tablet_manager.cc:1428] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:31:35.291069 21967 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:35.291333 21967 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.292608 21967 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
I20260806 12:31:35.297290 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.292920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:41841 (local address 127.6.152.131:35363)
0806 12:31:35.293034 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.293038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.293380 (+ 342us) server_negotiation.cc:407] Connection header received
0806 12:31:35.293422 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.293424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.293459 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.293512 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:35.293771 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.294461 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.296006 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.296604 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.296743 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.296793 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.297098 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.297169 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:31:35.297290 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.292842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:53437 (local address 127.6.152.130:42187)
0806 12:31:35.292956 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.292962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.293087 (+ 125us) server_negotiation.cc:407] Connection header received
0806 12:31:35.293197 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.293199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.293239 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.293305 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:35.293585 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.294363 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.296006 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.296604 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.296743 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.296793 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.297098 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.297169 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:31:35.297310 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.292774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42187 (local address 127.6.152.129:53437)
0806 12:31:35.292955 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.292971 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.293071 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.293327 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:35.293330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:35.293346 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:35.293452 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.293457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.294474 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.294477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.295863 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.295875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.296745 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.296749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.296903 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.296974 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 12:31:35.297057 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:31:35.297153 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":94}
I20260806 12:31:35.297327 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.292820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35363 (local address 127.6.152.129:41841)
0806 12:31:35.293077 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.293088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.293185 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.293555 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:35.293561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:35.293580 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:35.293673 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.293677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.294549 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.294552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.295863 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:35.295875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.296750 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.296754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:35.296917 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.296979 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 12:31:35.297057 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:31:35.297153 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":97,"thread_start_us":61,"threads_started":1}
I20260806 12:31:35.297964 22054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7047e9958844a76817eb1299cd50f1e" candidate_uuid: "a84e7f020baa48b0a32178ad171e46ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57eb4acf134530aae0d926611f6e44" is_pre_election: true
I20260806 12:31:35.297964 22189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7047e9958844a76817eb1299cd50f1e" candidate_uuid: "a84e7f020baa48b0a32178ad171e46ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118e53bedd0140c6adf15aec70d548f1" is_pre_election: true
I20260806 12:31:35.298242 22054 raft_consensus.cc:2454] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a84e7f020baa48b0a32178ad171e46ab in term 0.
I20260806 12:31:35.298242 22189 raft_consensus.cc:2454] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a84e7f020baa48b0a32178ad171e46ab in term 0.
I20260806 12:31:35.298774 21846 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 118e53bedd0140c6adf15aec70d548f1, a84e7f020baa48b0a32178ad171e46ab, ef57eb4acf134530aae0d926611f6e44; no voters:
I20260806 12:31:35.299067 21967 raft_consensus.cc:2781] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:31:35.299189 21967 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:31:35.299268 21967 raft_consensus.cc:3037] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:35.300319 21967 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.300684 21967 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 1 election: Requested vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
I20260806 12:31:35.301043 22189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7047e9958844a76817eb1299cd50f1e" candidate_uuid: "a84e7f020baa48b0a32178ad171e46ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118e53bedd0140c6adf15aec70d548f1"
I20260806 12:31:35.301043 22054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7047e9958844a76817eb1299cd50f1e" candidate_uuid: "a84e7f020baa48b0a32178ad171e46ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57eb4acf134530aae0d926611f6e44"
I20260806 12:31:35.301162 22054 raft_consensus.cc:3037] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:35.301162 22189 raft_consensus.cc:3037] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:35.302439 22189 raft_consensus.cc:2454] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a84e7f020baa48b0a32178ad171e46ab in term 1.
I20260806 12:31:35.302436 22054 raft_consensus.cc:2454] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a84e7f020baa48b0a32178ad171e46ab in term 1.
I20260806 12:31:35.302735 21848 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118e53bedd0140c6adf15aec70d548f1, a84e7f020baa48b0a32178ad171e46ab; no voters:
I20260806 12:31:35.302953 21967 raft_consensus.cc:2781] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:35.303207 21967 raft_consensus.cc:686] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 LEADER]: Becoming Leader. State: Replica: a84e7f020baa48b0a32178ad171e46ab, State: Running, Role: LEADER
I20260806 12:31:35.303476 21967 consensus_queue.cc:237] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.305913 21770 catalog_manager.cc:5495] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab reported cstate change: term changed from 0 to 1, leader changed from <none> to a84e7f020baa48b0a32178ad171e46ab (127.6.152.129). New cstate: current_term: 1 leader_uuid: "a84e7f020baa48b0a32178ad171e46ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } health_report { overall_health: HEALTHY } } }
W20260806 12:31:35.316327 21962 tablet.cc:2369] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:31:35.324592 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.320996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.6.152.129:39011)
0806 12:31:35.321128 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:35.321132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:35.321146 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:31:35.321262 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:35.321268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:35.321323 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:35.321432 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:31:35.321686 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.322216 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.323061 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:35.323377 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:35.323493 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:35.323801 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:31:35.324186 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:31:35.324321 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:35.324390 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 12:31:35.324455 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:31:35.328635 22054 raft_consensus.cc:1261] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Refusing update from remote peer a84e7f020baa48b0a32178ad171e46ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:35.328603 22189 raft_consensus.cc:1261] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Refusing update from remote peer a84e7f020baa48b0a32178ad171e46ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:35.329311 21967 consensus_queue.cc:1035] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:35.329537 22256 consensus_queue.cc:1035] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:35.337293 22259 mvcc.cc:205] Tried to move back new op lower bound from 7315535852859338752 to 7315535852765556736. Current Snapshot: MvccSnapshot[applied={T|T < 7315535852859338752}]
I20260806 12:31:35.337978 22261 mvcc.cc:205] Tried to move back new op lower bound from 7315535852859338752 to 7315535852765556736. Current Snapshot: MvccSnapshot[applied={T|T < 7315535852859338752}]
I20260806 12:31:35.346481 21770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:35.346621 21770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:35.348896 21770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Sending DeleteTablet for 3 replicas of tablet e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:35.349625 22168 tablet_service.cc:1514] Processing DeleteTablet for tablet e7047e9958844a76817eb1299cd50f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:35 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:49716
I20260806 12:31:35.349694 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21809
I20260806 12:31:35.349836 22033 tablet_service.cc:1514] Processing DeleteTablet for tablet e7047e9958844a76817eb1299cd50f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:35 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:51226
I20260806 12:31:35.351567 22269 tablet_replica.cc:330] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: stopping tablet replica
I20260806 12:31:35.351832 22269 raft_consensus.cc:2229] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:31:35.352072 22269 raft_consensus.cc:2258] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:35.352324 22268 tablet_replica.cc:330] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: stopping tablet replica
I20260806 12:31:35.352584 22268 raft_consensus.cc:2229] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:31:35.352906 22268 raft_consensus.cc:2258] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:35.353618 22269 ts_tablet_manager.cc:1905] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:31:35.356525 22268 ts_tablet_manager.cc:1905] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:31:35.357328 22269 ts_tablet_manager.cc:1918] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:35.357441 22269 log.cc:1196] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:35.357888 22269 ts_tablet_manager.cc:1939] T e7047e9958844a76817eb1299cd50f1e P ef57eb4acf134530aae0d926611f6e44: Deleting consensus metadata
I20260806 12:31:35.358740 22268 ts_tablet_manager.cc:1918] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:35.358876 22268 log.cc:1196] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:35.359200 21754 catalog_manager.cc:4841] TS ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): tablet e7047e9958844a76817eb1299cd50f1e (table test-table [id=cb58a6d3dba34cd1b660c06e8d5d0714]) successfully deleted
I20260806 12:31:35.359272 22268 ts_tablet_manager.cc:1939] T e7047e9958844a76817eb1299cd50f1e P 118e53bedd0140c6adf15aec70d548f1: Deleting consensus metadata
I20260806 12:31:35.360260 21756 catalog_manager.cc:4841] TS 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): tablet e7047e9958844a76817eb1299cd50f1e (table test-table [id=cb58a6d3dba34cd1b660c06e8d5d0714]) successfully deleted
W20260806 12:31:35.361828 21754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:39011 (error 108)
I20260806 12:31:35.362167 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21965
I20260806 12:31:35.363538 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.363176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:39011 (local address 127.0.0.1:32920)
0806 12:31:35.363322 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.363422 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 12:31:35.363723 21754 catalog_manager.cc:4584] TS a84e7f020baa48b0a32178ad171e46ab (127.6.152.129:39011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7047e9958844a76817eb1299cd50f1e: Network error: Client connection negotiation failed: client connection to 127.6.152.129:39011: connect: Connection refused (error 111)
I20260806 12:31:35.369479 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 22104
I20260806 12:31:35.375802 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 21733
I20260806 12:31:35.383091 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37149
--webserver_interface=127.6.152.190
--webserver_port=46459
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:35.492332 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:35.492569 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:35.492610 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:35.496518 22271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:35.496563 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:35.496582 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:35.496614 22271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:35.496628 22271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:35.501618 22271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=46459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:35.502553 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:35.503563 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:35.511650 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:35.511693 22271 server_base.cc:1034] running on GCE node
W20260806 12:31:35.511672 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:35.511732 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:35.512498 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:35.513504 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:35.514695 22271 hybrid_clock.cc:648] HybridClock initialized: now 1786019495514687 us; error 29 us; skew 500 ppm
Aug 06 12:31:35 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:35.517330 22271 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:35.518115 22271 webserver.cc:460] Webserver started at http://127.6.152.190:46459/ using document root <none> and password file <none>
I20260806 12:31:35.518585 22271 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:35.518643 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:35.522151 22271 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:35.524199 22286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.525336 22271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:35.525460 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "55283a3fa83f4e6abb774d221ab41256"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "f1be9789f59dfb4ec9420849e0f1145a"
server_key_iv: "4a253202befd15525e112b3cd3006178"
server_key_version: "encryptionkey@0"
I20260806 12:31:35.525965 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:35.536783 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:35.539553 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:35.539762 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:35.546880 22271 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37149
I20260806 12:31:35.547235 22339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37149 every 8 connection(s)
I20260806 12:31:35.548017 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:35.549499 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22271
I20260806 12:31:35.549948 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:39011
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:37149
--webserver_port=38705
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:35.552317 22340 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:31:35.553938 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap starting.
I20260806 12:31:35.571750 22340 log.cc:824] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:35.582614 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:31:35.582989 22340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256: Bootstrap complete.
I20260806 12:31:35.587067 22340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:35.587554 22340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55283a3fa83f4e6abb774d221ab41256, State: Initialized, Role: FOLLOWER
I20260806 12:31:35.588029 22340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:35.588172 22340 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:35.588258 22340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:35.588380 22340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:31:35.590197 22340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:35.590517 22340 leader_election.cc:302] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55283a3fa83f4e6abb774d221ab41256; no voters:
I20260806 12:31:35.590847 22340 leader_election.cc:288] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 12:31:35.590940 22344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 12:31:35.591166 22344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [term 3 LEADER]: Becoming Leader. State: Replica: 55283a3fa83f4e6abb774d221ab41256, State: Running, Role: LEADER
I20260806 12:31:35.591526 22344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } }
I20260806 12:31:35.592085 22340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:35.592960 22345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "55283a3fa83f4e6abb774d221ab41256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } } }
I20260806 12:31:35.593125 22345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:35.593567 22346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55283a3fa83f4e6abb774d221ab41256. Latest consensus state: current_term: 3 leader_uuid: "55283a3fa83f4e6abb774d221ab41256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55283a3fa83f4e6abb774d221ab41256" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37149 } } }
I20260806 12:31:35.593736 22346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55283a3fa83f4e6abb774d221ab41256 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:35.594048 22353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:35.599282 22353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:35.599900 22353 catalog_manager.cc:1259] Loaded cluster ID: c11e8c8b2ccb4401a78b303f77fc19f6
I20260806 12:31:35.599963 22353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 12:31:35.601019 22353 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1b936a09ad google::LogMessage::Fail() at ??:0
@ 0x7f1b936a388a google::LogMessage::SendToLog() at ??:0
@ 0x7f1b936a04de google::LogMessage::Flush() at ??:0
@ 0x7f1b936a1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1b9dfdffdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1b9dfe0d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1b9dfdd579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1b9e021640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1b958f23d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1b958f2c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1b958f43f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1b958e3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1b96f276db start_thread at ??:0
@ 0x7f1b9219e61f clone at ??:0
W20260806 12:31:35.690562 22342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:35.690831 22342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:35.690937 22342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:35.694929 22342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:35.695011 22342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:35.695103 22342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:31:35.700307 22342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:39011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=38705
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:35.701314 22342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:35.702301 22342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:35.712819 22383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:35.712932 22385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:35.712814 22382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:35.713572 22342 server_base.cc:1034] running on GCE node
I20260806 12:31:35.713963 22342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:35.714605 22342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:35.715791 22342 hybrid_clock.cc:648] HybridClock initialized: now 1786019495715775 us; error 54 us; skew 500 ppm
Aug 06 12:31:35 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:35.718470 22342 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:31:35.719543 22342 webserver.cc:460] Webserver started at http://127.6.152.129:38705/ using document root <none> and password file <none>
I20260806 12:31:35.720078 22342 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:35.720140 22342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:35.724208 22342 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:31:35.726351 22393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.727705 22342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:35.727797 22342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "a84e7f020baa48b0a32178ad171e46ab"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "64d92076300383003e2445a99f49aa39"
server_key_iv: "e24da1e09808caf9cd3840a3bcd52a7c"
server_key_version: "encryptionkey@0"
I20260806 12:31:35.728343 22342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:35.754848 22342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:35.759193 22342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:35.759415 22342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:35.760171 22342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:35.761718 22400 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:31:35.767171 22342 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:31:35.767258 22342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 12:31:35.767338 22342 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:31:35.769170 22342 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:31:35.769244 22342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 12:31:35.769546 22400 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Bootstrap starting.
I20260806 12:31:35.780232 22342 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:39011
I20260806 12:31:35.781176 22342 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:31:35.782377 22508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:39011 every 8 connection(s)
I20260806 12:31:35.787514 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22342
I20260806 12:31:35.787956 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:42187
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:37149
--webserver_port=38291
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:35.794317 22400 log.cc:824] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:35.801411 22400 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:31:35.801784 22400 tablet_bootstrap.cc:492] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Bootstrap complete.
I20260806 12:31:35.802997 22400 ts_tablet_manager.cc:1397] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.005s
I20260806 12:31:35.805459 22400 raft_consensus.cc:348] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.805894 22400 raft_consensus.cc:729] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84e7f020baa48b0a32178ad171e46ab, State: Initialized, Role: FOLLOWER
I20260806 12:31:35.806399 22400 consensus_queue.cc:260] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:35.807405 22400 ts_tablet_manager.cc:1428] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
W20260806 12:31:35.826858 22511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:31:35.783407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:35053)
0806 12:31:35.783929 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.783983 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:35.784816 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:35.826121 (+ 41305us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":159,"threads_started":1}
W20260806 12:31:35.827394 22509 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:37149 (0 consecutive failures): Network error: Failed to ping master at 127.6.152.190:37149: Client connection negotiation failed: client connection to 127.6.152.190:37149: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 12:31:35.828150 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.827902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:33129)
0806 12:31:35.828030 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.828096 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 12:31:35.828783 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.828584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:43155)
0806 12:31:35.828687 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.828738 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 12:31:35.828945 22509 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:31:35.913393 22512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:35.913666 22512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:35.913743 22512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:35.917851 22512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:35.917941 22512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:35.918047 22512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:31:35.923338 22512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:42187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=38291
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:35.924350 22512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:35.925382 22512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:35.935191 22522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:35.935207 22520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:35.935350 22512 server_base.cc:1034] running on GCE node
W20260806 12:31:35.935235 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:35.935812 22512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:35.936482 22512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:35.937647 22512 hybrid_clock.cc:648] HybridClock initialized: now 1786019495937609 us; error 50 us; skew 500 ppm
Aug 06 12:31:35 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:35.940419 22512 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:31:35.941193 22512 webserver.cc:460] Webserver started at http://127.6.152.130:38291/ using document root <none> and password file <none>
I20260806 12:31:35.941705 22512 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:35.941764 22512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:35.945403 22512 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 12:31:35.947307 22529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.948472 22512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:31:35.948626 22512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "ef57eb4acf134530aae0d926611f6e44"
format_stamp: "Formatted at 2026-08-06 12:31:33 on dist-test-slave-1xrh"
server_key: "96ae35b80d943cb0343f67c0f008f53a"
server_key_iv: "3b2581f6c84e836a7dac89af154131c1"
server_key_version: "encryptionkey@0"
I20260806 12:31:35.949133 22512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:35.960216 22512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:35.962741 22512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:35.962924 22512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:35.963546 22512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:35.964315 22512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:35.964365 22512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.964429 22512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:35.964468 22512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:35.973151 22512 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:42187
I20260806 12:31:35.973183 22645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:42187 every 8 connection(s)
I20260806 12:31:35.974082 22512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:31:35.975941 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22512
I20260806 12:31:35.976431 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:35363
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:37149
--webserver_port=39687
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:31:35.977916 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.976309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:45265)
0806 12:31:35.977094 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.977200 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":151,"threads_started":1}
W20260806 12:31:35.978420 22646 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:37149 (0 consecutive failures): Network error: Failed to ping master at 127.6.152.190:37149: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
I20260806 12:31:35.979108 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.978807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:59463)
0806 12:31:35.978946 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.979021 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 12:31:35.979835 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:35.979642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:46423)
0806 12:31:35.979744 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:35.979787 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 12:31:35.980046 22646 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:31:36.086220 22649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:36.086488 22649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:36.086565 22649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:36.090600 22649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:36.090690 22649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:36.090781 22649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:31:36.095938 22649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:35363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=39687
--tserver_master_addrs=127.6.152.190:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:36.096978 22649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:36.098023 22649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:36.107445 22655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:36.107501 22657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:36.107489 22654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:36.107636 22649 server_base.cc:1034] running on GCE node
I20260806 12:31:36.108085 22649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:36.108757 22649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:36.109961 22649 hybrid_clock.cc:648] HybridClock initialized: now 1786019496109946 us; error 37 us; skew 500 ppm
Aug 06 12:31:36 dist-test-slave-1xrh krb5kdc[21152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019496, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:36.112716 22649 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:31:36.113543 22649 webserver.cc:460] Webserver started at http://127.6.152.131:39687/ using document root <none> and password file <none>
I20260806 12:31:36.114074 22649 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:36.114132 22649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:36.117702 22649 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:31:36.119616 22665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:36.120795 22649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:36.120890 22649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "118e53bedd0140c6adf15aec70d548f1"
format_stamp: "Formatted at 2026-08-06 12:31:34 on dist-test-slave-1xrh"
server_key: "8b5e81284e998dd9f8d26849f34e86e1"
server_key_iv: "cdcb9edd002dec14cb2754f204c70f29"
server_key_version: "encryptionkey@0"
I20260806 12:31:36.121381 22649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:36.147598 22649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:36.150264 22649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:36.150408 22649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:36.150918 22649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:36.151696 22649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:36.151787 22649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:36.151859 22649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:36.151913 22649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:36.161296 22649 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35363
I20260806 12:31:36.161334 22782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35363 every 8 connection(s)
I20260806 12:31:36.162250 22649 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:31:36.164209 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22649
I20260806 12:31:36.166184 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:36.164388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:41811)
0806 12:31:36.165372 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:36.165503 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":613,"threads_started":1}
W20260806 12:31:36.166651 22783 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:37149 (0 consecutive failures): Network error: Failed to ping master at 127.6.152.190:37149: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
I20260806 12:31:36.167271 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:36.167018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:58365)
0806 12:31:36.167158 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:36.167216 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 12:31:36.167907 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:36.167730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:51223)
0806 12:31:36.167818 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:36.167859 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 12:31:36.168149 22783 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 12:31:36.829942 22787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:36.829554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:33067)
0806 12:31:36.829795 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:36.829849 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260806 12:31:36.981055 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:36.980618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:56881)
0806 12:31:36.980879 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:36.980948 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260806 12:31:37.169040 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.168642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:40511)
0806 12:31:37.168890 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:37.168944 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":84,"threads_started":1}
I20260806 12:31:37.672118 22790 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:37.672303 22790 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:37.673378 22790 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
Aug 06 12:31:37 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.130@KRBTEST.COM
Aug 06 12:31:37 dist-test-slave-1xrh krb5kdc[21152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019495, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.131@KRBTEST.COM
I20260806 12:31:37.691326 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.673523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:42187 (local address 127.6.152.129:34139)
0806 12:31:37.673825 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:37.673841 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:37.673949 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:37.675605 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:37.675611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:37.676165 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:37.676637 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:37.676656 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.678608 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.678612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:37.679973 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:37.679983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.680832 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.680836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:37.681016 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:37.681881 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:37.681893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:37.684352 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:37.687803 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:37.687813 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:37.687830 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:37.688265 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:37.688809 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:37.688826 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:37.688835 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:37.689096 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:37.690021 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:37.690042 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:37.690490 (+ 448us) client_negotiation.cc:769] Sending connection context
0806 12:31:37.690689 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 12:31:37.691023 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":31,"spinlock_wait_cycles":1920,"thread_start_us":64,"threads_started":1}
I20260806 12:31:37.692732 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.673717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:34139 (local address 127.6.152.130:42187)
0806 12:31:37.674072 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:37.674090 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:37.674147 (+ 57us) server_negotiation.cc:407] Connection header received
0806 12:31:37.674811 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:37.674817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:37.675243 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:37.675556 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:37.676876 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.678425 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.680101 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.680704 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.680893 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:37.684631 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:37.684639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:37.684658 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:37.684700 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:37.687655 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:37.688432 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:37.688441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:37.688450 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:37.688614 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:37.689244 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:37.689248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:37.689250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:37.689859 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:37.692023 (+ 2164us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:37.692223 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 12:31:37.692550 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":63,"threads_started":1}
W20260806 12:31:37.695763 22395 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:37.696574 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.673658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35363 (local address 127.6.152.129:60971)
0806 12:31:37.674118 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:37.674133 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:37.674227 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:37.675606 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:37.675612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:37.676165 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:37.676631 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:37.676668 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.678610 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.678614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:37.679955 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:37.679984 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.680992 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.680996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:37.681081 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:37.681805 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:37.681846 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:37.686372 (+ 4526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:37.694225 (+ 7853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:37.694230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:37.694235 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:37.694485 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:37.695022 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:37.695035 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:37.695040 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:37.695185 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:37.695897 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:37.695900 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:37.696013 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 12:31:37.696126 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 12:31:37.696294 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":303,"spinlock_wait_cycles":13824,"thread_start_us":77,"threads_started":1}
I20260806 12:31:37.697495 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.673877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:60971 (local address 127.6.152.131:35363)
0806 12:31:37.674241 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:37.674256 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:37.674299 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:31:37.674924 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:37.674929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:37.675292 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:37.675562 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:37.676866 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.678425 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.680129 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:37.680863 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:37.681051 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:37.686600 (+ 5549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:37.686610 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:37.686628 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:37.686669 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:37.689161 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:37.694701 (+ 5540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:37.694707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:37.694713 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:37.694896 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:37.695308 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:37.695313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:37.695315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:37.695778 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:37.695934 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:37.696900 (+ 966us) server_negotiation.cc:299] Negotiation successful
0806 12:31:37.697225 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":59,"threads_started":1}
W20260806 12:31:37.700060 22397 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:37.700170 22397 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a84e7f020baa48b0a32178ad171e46ab; no voters: 118e53bedd0140c6adf15aec70d548f1, ef57eb4acf134530aae0d926611f6e44
I20260806 12:31:37.700469 22790 raft_consensus.cc:2726] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:31:37.830924 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.830655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:34505)
0806 12:31:37.830778 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:37.830837 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 12:31:37.982319 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:37.981893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:57635)
0806 12:31:37.982165 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:37.982215 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260806 12:31:38.170047 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:38.169667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:33471)
0806 12:31:38.169891 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:38.169939 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260806 12:31:38.832311 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:38.831810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:60107)
0806 12:31:38.832125 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:38.832179 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":123,"threads_started":1}
I20260806 12:31:38.983430 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:38.983008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:36243)
0806 12:31:38.983286 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:38.983338 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":74,"threads_started":1}
I20260806 12:31:39.171115 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:39.170728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:53323)
0806 12:31:39.170970 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:39.171025 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260806 12:31:39.720302 22800 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:39.720461 22800 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:39.720923 22800 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
W20260806 12:31:39.721840 22397 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
W20260806 12:31:39.721933 22395 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:39.722046 22395 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a84e7f020baa48b0a32178ad171e46ab; no voters: 118e53bedd0140c6adf15aec70d548f1, ef57eb4acf134530aae0d926611f6e44
I20260806 12:31:39.722246 22800 raft_consensus.cc:2726] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:31:39.833384 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:39.832988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:42071)
0806 12:31:39.833238 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:39.833289 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":91,"threads_started":1}
I20260806 12:31:39.984639 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:39.984186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:35035)
0806 12:31:39.984457 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:39.984515 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
I20260806 12:31:40.172500 22803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:40.171908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:55159)
0806 12:31:40.172281 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:40.172375 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":94,"threads_started":1}
I20260806 12:31:40.834792 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:40.834328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:38249)
0806 12:31:40.834640 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:40.834695 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":91,"threads_started":1}
I20260806 12:31:40.985865 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:40.985427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:36039)
0806 12:31:40.985687 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:40.985746 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":91,"threads_started":1}
I20260806 12:31:41.173638 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:41.173249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:56249)
0806 12:31:41.173490 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:41.173548 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260806 12:31:41.835981 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:41.835524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:59191)
0806 12:31:41.835809 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:41.835871 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":77,"threads_started":1}
I20260806 12:31:41.987170 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:41.986711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:59931)
0806 12:31:41.986974 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:41.987032 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
I20260806 12:31:42.174748 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:42.174323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:32851)
0806 12:31:42.174604 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:42.174660 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":74,"threads_started":1}
I20260806 12:31:42.744894 22810 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:42.745026 22810 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:42.745440 22810 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
W20260806 12:31:42.746272 22397 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
W20260806 12:31:42.746425 22395 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:42.746507 22395 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a84e7f020baa48b0a32178ad171e46ab; no voters: 118e53bedd0140c6adf15aec70d548f1, ef57eb4acf134530aae0d926611f6e44
I20260806 12:31:42.746694 22810 raft_consensus.cc:2726] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:31:42.837361 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:42.836815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:43061)
0806 12:31:42.837188 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:42.837261 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":168,"threads_started":1}
I20260806 12:31:42.988446 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:42.987968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:51411)
0806 12:31:42.988264 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:42.988321 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":76,"threads_started":1}
I20260806 12:31:43.175976 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:43.175530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:41583)
0806 12:31:43.175832 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:43.175889 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":67,"threads_started":1}
I20260806 12:31:43.838521 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:43.838095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:35961)
0806 12:31:43.838358 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:43.838416 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
I20260806 12:31:43.989709 22815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:43.989304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:39623)
0806 12:31:43.989557 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:43.989612 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":85,"threads_started":1}
I20260806 12:31:44.177210 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:44.176773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:49271)
0806 12:31:44.177066 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:44.177122 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
I20260806 12:31:44.839689 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:44.839276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:58229)
0806 12:31:44.839534 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:44.839591 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
I20260806 12:31:44.991053 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:44.990462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:42917)
0806 12:31:44.990900 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:44.990959 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":276,"threads_started":1}
I20260806 12:31:45.178349 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:45.177888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:56915)
0806 12:31:45.178177 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:45.178237 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":93,"threads_started":1}
I20260806 12:31:45.840942 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:45.840540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:51705)
0806 12:31:45.840795 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:45.840848 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
I20260806 12:31:45.992372 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:45.991947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:47107)
0806 12:31:45.992221 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:45.992278 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
I20260806 12:31:46.179687 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:46.179274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:49435)
0806 12:31:46.179542 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:46.179596 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":89,"threads_started":1}
I20260806 12:31:46.842118 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:46.841691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:38187)
0806 12:31:46.841955 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:46.842011 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260806 12:31:46.993687 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:46.993232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:38435)
0806 12:31:46.993526 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:46.993587 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
I20260806 12:31:47.140859 22826 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:47.140985 22826 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:47.141575 22826 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
W20260806 12:31:47.142340 22397 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
W20260806 12:31:47.142438 22395 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:47.142549 22395 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a84e7f020baa48b0a32178ad171e46ab; no voters: 118e53bedd0140c6adf15aec70d548f1, ef57eb4acf134530aae0d926611f6e44
I20260806 12:31:47.142702 22826 raft_consensus.cc:2726] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:31:47.180744 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:47.180364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:37689)
0806 12:31:47.180601 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:47.180655 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260806 12:31:47.843324 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:47.842895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:53205)
0806 12:31:47.843175 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:47.843229 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":107,"threads_started":1}
I20260806 12:31:47.994985 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:47.994455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:56163)
0806 12:31:47.994808 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:47.994870 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":127,"threads_started":1}
I20260806 12:31:48.181869 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:48.181470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:33247)
0806 12:31:48.181719 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:48.181773 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260806 12:31:48.844506 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:48.844079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:59957)
0806 12:31:48.844356 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:48.844410 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":100,"threads_started":1}
I20260806 12:31:48.996157 22833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:48.995753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:38065)
0806 12:31:48.996002 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:48.996059 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260806 12:31:49.183068 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:49.182674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:47587)
0806 12:31:49.182920 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:49.182975 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260806 12:31:49.532545 22835 raft_consensus.cc:482] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:49.532675 22835 raft_consensus.cc:504] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57eb4acf134530aae0d926611f6e44" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 42187 } } peers { permanent_uuid: "118e53bedd0140c6adf15aec70d548f1" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35363 } } peers { permanent_uuid: "a84e7f020baa48b0a32178ad171e46ab" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 39011 } }
I20260806 12:31:49.533124 22835 leader_election.cc:288] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187), 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363)
W20260806 12:31:49.534083 22397 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 118e53bedd0140c6adf15aec70d548f1 (127.6.152.131:35363): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
W20260806 12:31:49.534198 22395 leader_election.cc:341] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef57eb4acf134530aae0d926611f6e44 (127.6.152.130:42187): Not found: Tablet not found: e7047e9958844a76817eb1299cd50f1e
I20260806 12:31:49.534269 22395 leader_election.cc:302] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a84e7f020baa48b0a32178ad171e46ab; no voters: 118e53bedd0140c6adf15aec70d548f1, ef57eb4acf134530aae0d926611f6e44
I20260806 12:31:49.534428 22835 raft_consensus.cc:2726] T e7047e9958844a76817eb1299cd50f1e P a84e7f020baa48b0a32178ad171e46ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:31:49.845739 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:49.845239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:58711)
0806 12:31:49.845587 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:49.845644 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":76,"threads_started":1}
I20260806 12:31:49.997336 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:49.996943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:51009)
0806 12:31:49.997186 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:49.997241 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260806 12:31:50.184309 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:50.183858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:37179)
0806 12:31:50.184144 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:50.184215 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":114,"threads_started":1}
I20260806 12:31:50.847245 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:50.846773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.129:46515)
0806 12:31:50.847025 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:50.847082 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260806 12:31:50.998575 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:50.998167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.130:46991)
0806 12:31:50.998424 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:50.998478 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260806 12:31:51.172441 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 22342
I20260806 12:31:51.180534 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 22512
I20260806 12:31:51.185562 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:51.185164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37149 (local address 127.6.152.131:59907)
0806 12:31:51.185417 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:51.185470 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
I20260806 12:31:51.187669 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 22649
2026-08-06T12:31:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19079 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:31:51 dist-test-slave-1xrh krb5kdc[22845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:31:51 dist-test-slave-1xrh krb5kdc[22845](info): set up 2 sockets
Aug 06 12:31:51 dist-test-slave-1xrh krb5kdc[22845](info): commencing operation
krb5kdc: starting...
W20260806 12:31:52.256531 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:31:52 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:31:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:31:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:31:52.389964 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38245
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:38245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:52.503048 22858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:52.503307 22858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:52.503346 22858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:52.507282 22858 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 12:31:52.507330 22858 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 12:31:52.507346 22858 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 12:31:52.507361 22858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:31:52.507378 22858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:52.507397 22858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:52.507412 22858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:31:52.507432 22858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:31:52.512477 22858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:38245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:52.513396 22858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:52.514323 22858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:52.522990 22866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:52.523000 22863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:52.523047 22864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:52.523193 22858 server_base.cc:1034] running on GCE node
I20260806 12:31:52.523916 22858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:52.524964 22858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:52.526145 22858 hybrid_clock.cc:648] HybridClock initialized: now 1786019512526125 us; error 31 us; skew 500 ppm
Aug 06 12:31:52 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019512, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:52.529075 22858 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:31:52.529959 22858 webserver.cc:460] Webserver started at http://127.6.152.190:40749/ using document root <none> and password file <none>
I20260806 12:31:52.530530 22858 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:52.530627 22858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:52.530867 22858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:52.532960 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "d925806e76ef4db29da179ede05c9a91"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "6eb1b8cdad2e3614313982ee772dacb0"
server_key_iv: "6fbd66cbf2ae26c202a455b21b413c7f"
server_key_version: "encryptionkey@0"
I20260806 12:31:52.533536 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "d925806e76ef4db29da179ede05c9a91"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "6eb1b8cdad2e3614313982ee772dacb0"
server_key_iv: "6fbd66cbf2ae26c202a455b21b413c7f"
server_key_version: "encryptionkey@0"
I20260806 12:31:52.537295 22858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:31:52.539633 22873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:52.540796 22858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:31:52.540940 22858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "d925806e76ef4db29da179ede05c9a91"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "6eb1b8cdad2e3614313982ee772dacb0"
server_key_iv: "6fbd66cbf2ae26c202a455b21b413c7f"
server_key_version: "encryptionkey@0"
I20260806 12:31:52.541047 22858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:52.582705 22858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:52.586208 22858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:52.586441 22858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:52.593752 22858 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:38245
I20260806 12:31:52.594055 22928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:38245 every 8 connection(s)
I20260806 12:31:52.594732 22858 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:31:52.597618 22929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:52.597640 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22858
I20260806 12:31:52.597754 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:31:52.598109 6754 external_mini_cluster.cc:1428] Setting key 449b92e787041c3e1b13a8c45d07869a
I20260806 12:31:52.604585 22929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap starting.
I20260806 12:31:52.607425 22929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Neither blocks nor log segments found. Creating new log.
Aug 06 12:31:52 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:52.608234 22929 log.cc:824] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:52.610572 22929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: No bootstrap required, opened a new log
I20260806 12:31:52.613282 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:52.599536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40964 (local address 127.6.152.190:38245)
0806 12:31:52.599984 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:52.600000 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:52.600036 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:31:52.600882 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:52.600932 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:52.601244 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:52.601529 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:52.602459 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:52.603488 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:52.604659 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:52.605164 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:52.605278 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:52.608135 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:52.608141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:52.608156 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:52.608186 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:52.610106 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:52.610582 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:52.610591 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:52.610598 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:52.610735 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:52.611007 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:52.611010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:52.611011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:52.611547 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:52.611742 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:52.611956 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 12:31:52.612251 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":169,"threads_started":1}
I20260806 12:31:52.615056 22929 raft_consensus.cc:348] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:31:52.615358 22929 raft_consensus.cc:374] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:52.615430 22929 raft_consensus.cc:729] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d925806e76ef4db29da179ede05c9a91, State: Initialized, Role: FOLLOWER
I20260806 12:31:52.615826 22929 consensus_queue.cc:260] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:31:52.615948 22929 raft_consensus.cc:388] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:31:52.616040 22929 raft_consensus.cc:482] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:31:52.616139 22929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:52.616998 22929 raft_consensus.cc:504] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:31:52.617342 22929 leader_election.cc:302] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d925806e76ef4db29da179ede05c9a91; no voters:
I20260806 12:31:52.617632 22929 leader_election.cc:288] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:31:52.617873 22934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:52.618093 22934 raft_consensus.cc:686] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 LEADER]: Becoming Leader. State: Replica: d925806e76ef4db29da179ede05c9a91, State: Running, Role: LEADER
I20260806 12:31:52.618414 22934 consensus_queue.cc:237] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:31:52.618924 22929 sys_catalog.cc:564] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:31:52.619626 22936 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d925806e76ef4db29da179ede05c9a91. Latest consensus state: current_term: 1 leader_uuid: "d925806e76ef4db29da179ede05c9a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } } }
I20260806 12:31:52.619799 22936 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:52.620085 22935 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d925806e76ef4db29da179ede05c9a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } } }
I20260806 12:31:52.620184 22935 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: This master's current role is: LEADER
I20260806 12:31:52.620473 22943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:31:52.623580 22943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:31:52.628871 22943 catalog_manager.cc:1347] Generated new cluster ID: 73b993b7b35645969bcb60968cf53d43
I20260806 12:31:52.628971 22943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:31:52.641561 22943 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:31:52.642743 22943 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:31:52.659168 22943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Generated new TSK 0
I20260806 12:31:52.660000 22943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:31:52.691759 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:38245
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:52.803854 22955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:52.804152 22955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:52.804226 22955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:52.808207 22955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:52.808279 22955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:52.808362 22955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:31:52.813501 22955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:52.814482 22955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:52.815531 22955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:52.825383 22963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:52.825484 22960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:52.825383 22961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:52.825451 22955 server_base.cc:1034] running on GCE node
I20260806 12:31:52.826040 22955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:52.826681 22955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:52.827869 22955 hybrid_clock.cc:648] HybridClock initialized: now 1786019512827855 us; error 25 us; skew 500 ppm
Aug 06 12:31:52 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019512, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:52.830585 22955 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:31:52.831453 22955 webserver.cc:460] Webserver started at http://127.6.152.129:38813/ using document root <none> and password file <none>
I20260806 12:31:52.831985 22955 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:52.832089 22955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:52.832319 22955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:52.834241 22955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "736ac31fd2bd4acd9161c033535c0d20"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "89a1933c9a14631daf847ff0ede27f31"
server_key_iv: "65f0f2efd502949dd46fa4f6c0849946"
server_key_version: "encryptionkey@0"
I20260806 12:31:52.834808 22955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "736ac31fd2bd4acd9161c033535c0d20"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "89a1933c9a14631daf847ff0ede27f31"
server_key_iv: "65f0f2efd502949dd46fa4f6c0849946"
server_key_version: "encryptionkey@0"
I20260806 12:31:52.838588 22955 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 12:31:52.840881 22970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:52.841993 22955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:52.842134 22955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "736ac31fd2bd4acd9161c033535c0d20"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "89a1933c9a14631daf847ff0ede27f31"
server_key_iv: "65f0f2efd502949dd46fa4f6c0849946"
server_key_version: "encryptionkey@0"
I20260806 12:31:52.842243 22955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:52.861176 22955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:52.864084 22955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:52.864276 22955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:52.864840 22955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:52.865640 22955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:52.865715 22955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:52.865784 22955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:52.865828 22955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:52.875218 22955 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:34393
I20260806 12:31:52.875238 23086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:34393 every 8 connection(s)
I20260806 12:31:52.876106 22955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:31:52.878144 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 22955
I20260806 12:31:52.878290 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:31:52.878568 6754 external_mini_cluster.cc:1428] Setting key a38bb916b03e493785ae55dac7c8551b
Aug 06 12:31:52 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019512, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:31:52.890070 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:52.877845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49019 (local address 127.6.152.190:38245)
0806 12:31:52.878013 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:52.878018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:52.879208 (+ 1190us) server_negotiation.cc:407] Connection header received
0806 12:31:52.880238 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:52.880241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:52.880293 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:52.880377 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:52.881420 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:52.882212 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:52.883424 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:52.883670 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:52.883782 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:52.886599 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:52.886602 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:52.886604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:52.886627 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:52.888095 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:52.888711 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:52.888713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:52.888715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:52.888790 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:52.889102 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:52.889104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:52.889105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:52.889269 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:52.889343 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:52.889844 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 12:31:52.889951 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 12:31:52.890846 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:52.878279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:49019)
0806 12:31:52.879030 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:52.879081 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:52.880046 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:52.880492 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:52.880498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:52.880830 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:52.881265 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:52.881277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:52.882347 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:52.882352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:52.883297 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:52.883307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:52.883840 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:52.883845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:52.884017 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:52.884592 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:52.884608 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:52.886430 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:52.888217 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:52.888227 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:52.888243 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:52.888610 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:52.888878 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:52.888880 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:52.888882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:52.889011 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:52.889370 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:52.889377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:52.889611 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 12:31:52.889764 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 12:31:52.890017 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":147,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:31:52.892185 23087 heartbeater.cc:347] Connected to a master server at 127.6.152.190:38245
I20260806 12:31:52.892396 23087 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:52.892870 23087 heartbeater.cc:511] Master 127.6.152.190:38245 requested a full tablet report, sending...
I20260806 12:31:52.894384 22890 ts_manager.cc:194] Registered new tserver with Master: 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393)
I20260806 12:31:52.895628 22890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:49019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:31:52.907720 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:38245
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:53.017458 23092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:53.017701 23092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:53.017745 23092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:53.021836 23092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:53.021888 23092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:53.021999 23092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:31:53.027092 23092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:53.028098 23092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:53.029093 23092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:53.038707 23100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:53.038707 23097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:53.038786 23092 server_base.cc:1034] running on GCE node
W20260806 12:31:53.038739 23098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:53.039345 23092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:53.039987 23092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:53.041216 23092 hybrid_clock.cc:648] HybridClock initialized: now 1786019513041201 us; error 27 us; skew 500 ppm
Aug 06 12:31:53 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:53.044242 23092 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:31:53.045131 23092 webserver.cc:460] Webserver started at http://127.6.152.130:40493/ using document root <none> and password file <none>
I20260806 12:31:53.045743 23092 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:53.045848 23092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:53.046077 23092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:53.048118 23092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "419532933013415da09839e93129881e"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "ec57b580915b93a2682aa3c3d19c3bd5"
server_key_iv: "36d2f5f3dbf62a3e0fff70cd82e9ff5c"
server_key_version: "encryptionkey@0"
I20260806 12:31:53.048681 23092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "419532933013415da09839e93129881e"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "ec57b580915b93a2682aa3c3d19c3bd5"
server_key_iv: "36d2f5f3dbf62a3e0fff70cd82e9ff5c"
server_key_version: "encryptionkey@0"
I20260806 12:31:53.052219 23092 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 12:31:53.054643 23107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:53.055794 23092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:53.055930 23092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "419532933013415da09839e93129881e"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "ec57b580915b93a2682aa3c3d19c3bd5"
server_key_iv: "36d2f5f3dbf62a3e0fff70cd82e9ff5c"
server_key_version: "encryptionkey@0"
I20260806 12:31:53.056041 23092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:53.076233 23092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:53.079605 23092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:53.079823 23092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:53.080433 23092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:53.081240 23092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:53.081313 23092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:53.081384 23092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:53.081441 23092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:53.090840 23092 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:40579
I20260806 12:31:53.090878 23225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:40579 every 8 connection(s)
I20260806 12:31:53.091804 23092 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:31:53.093719 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23092
I20260806 12:31:53.093844 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:31:53.094187 6754 external_mini_cluster.cc:1428] Setting key c67d9faabb71b988420089e9fbb611ff
Aug 06 12:31:53 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
I20260806 12:31:53.107189 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.093642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:40453 (local address 127.6.152.190:38245)
0806 12:31:53.093806 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.093812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.095421 (+ 1609us) server_negotiation.cc:407] Connection header received
0806 12:31:53.096406 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.096410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.096464 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.096551 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:53.097876 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.098401 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.100213 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.100470 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.100573 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.103507 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:53.103510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:53.103512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:53.103535 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:53.105110 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:53.105636 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:53.105645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:53.105646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:53.105719 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:53.106068 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:53.106071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:53.106073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:53.106253 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:53.106366 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.106863 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.106992 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:31:53.108145 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.094157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:40453)
0806 12:31:53.095278 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.095322 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.096203 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.096745 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.096752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.097104 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:53.097706 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.097728 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.098548 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.098551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.100084 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.100100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.100658 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.100664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.100886 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.101514 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:53.101551 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:53.103302 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:53.105233 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:53.105239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:53.105253 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:53.105522 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:53.105814 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:53.105817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:53.105819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:53.105963 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:53.106366 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:53.106373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:53.106621 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.106802 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.107115 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"thread_start_us":659,"threads_started":1}
I20260806 12:31:53.109383 23226 heartbeater.cc:347] Connected to a master server at 127.6.152.190:38245
I20260806 12:31:53.109652 23226 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:53.110267 23226 heartbeater.cc:511] Master 127.6.152.190:38245 requested a full tablet report, sending...
I20260806 12:31:53.111387 22890 ts_manager.cc:194] Registered new tserver with Master: 419532933013415da09839e93129881e (127.6.152.130:40579)
I20260806 12:31:53.112049 22890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:40453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:31:53.126087 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:38245
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:31:53.237368 23231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:31:53.237628 23231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:31:53.237697 23231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:31:53.241624 23231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:31:53.241709 23231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:31:53.241796 23231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:31:53.247175 23231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:31:53.248117 23231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:31:53.249105 23231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:31:53.258545 23237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:53.258520 23236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:31:53.258625 23240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:31:53.258715 23231 server_base.cc:1034] running on GCE node
I20260806 12:31:53.259277 23231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:31:53.259984 23231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:31:53.261164 23231 hybrid_clock.cc:648] HybridClock initialized: now 1786019513261149 us; error 30 us; skew 500 ppm
Aug 06 12:31:53 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:31:53.264006 23231 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:31:53.264889 23231 webserver.cc:460] Webserver started at http://127.6.152.131:34403/ using document root <none> and password file <none>
I20260806 12:31:53.265384 23231 fs_manager.cc:359] Metadata directory not provided
I20260806 12:31:53.265448 23231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:31:53.265599 23231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:31:53.267508 23231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "667a68c7828a4fddbcca08c2333b10a5"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "3fdd83035532e1a1aa6ef4194bbed3d9"
server_key_iv: "debd4fb719ee576b225d7e99fb685808"
server_key_version: "encryptionkey@0"
I20260806 12:31:53.268018 23231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "667a68c7828a4fddbcca08c2333b10a5"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "3fdd83035532e1a1aa6ef4194bbed3d9"
server_key_iv: "debd4fb719ee576b225d7e99fb685808"
server_key_version: "encryptionkey@0"
I20260806 12:31:53.271586 23231 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:31:53.273808 23247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:53.274938 23231 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:31:53.275071 23231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "667a68c7828a4fddbcca08c2333b10a5"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "3fdd83035532e1a1aa6ef4194bbed3d9"
server_key_iv: "debd4fb719ee576b225d7e99fb685808"
server_key_version: "encryptionkey@0"
I20260806 12:31:53.275228 23231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:31:53.295344 23231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:31:53.298553 23231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:31:53.298753 23231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:31:53.299382 23231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:31:53.300168 23231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:31:53.300237 23231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:53.300303 23231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:31:53.300356 23231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:31:53.309278 23231 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:44807
I20260806 12:31:53.309309 23364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:44807 every 8 connection(s)
I20260806 12:31:53.310397 23231 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:31:53.311967 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23231
I20260806 12:31:53.312108 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:31:53.312477 6754 external_mini_cluster.cc:1428] Setting key 15f7a9297f18cb8b8044de336194f9f3
Aug 06 12:31:53 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019513, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:31:53.326340 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.312446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:53343 (local address 127.6.152.190:38245)
0806 12:31:53.314336 (+ 1890us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.314341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.314587 (+ 246us) server_negotiation.cc:407] Connection header received
0806 12:31:53.315718 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.315722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.315777 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.315877 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:53.317515 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.318496 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.320108 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.320397 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.320542 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.322868 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:53.322872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:53.322875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:53.322902 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:53.324166 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:53.324683 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:53.324688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:53.324690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:53.324774 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:53.325123 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:53.325126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:53.325128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:53.325381 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:53.325473 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.326022 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.326145 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1804}
I20260806 12:31:53.327226 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.312922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:53343)
0806 12:31:53.314437 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.314492 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.315517 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.316082 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.316103 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.316622 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:31:53.317334 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.317354 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.318649 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.318652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.319989 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.320000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.320578 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.320582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.320771 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.321289 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:31:53.321315 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:31:53.322703 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:31:53.324286 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:53.324292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:53.324324 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:53.324575 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:53.324875 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:31:53.324879 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:31:53.324881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:31:53.325022 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:31:53.325474 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:31:53.325481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:31:53.325761 (+ 280us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.325979 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.326280 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1261,"thread_start_us":139,"threads_started":1}
I20260806 12:31:53.328315 23365 heartbeater.cc:347] Connected to a master server at 127.6.152.190:38245
I20260806 12:31:53.328524 23365 heartbeater.cc:464] Registering TS with master...
I20260806 12:31:53.328955 23365 heartbeater.cc:511] Master 127.6.152.190:38245 requested a full tablet report, sending...
I20260806 12:31:53.329861 22890 ts_manager.cc:194] Registered new tserver with Master: 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807)
I20260806 12:31:53.330379 22890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:53343
I20260806 12:31:53.338202 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:31:53.347878 22932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.340300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.6.152.190:38245)
0806 12:31:53.340410 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.340413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.340626 (+ 213us) server_negotiation.cc:407] Connection header received
0806 12:31:53.340696 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.340699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.340735 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.340799 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:31:53.341418 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.341926 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.342766 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.342981 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.343059 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.344584 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:31:53.344588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:31:53.344589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:31:53.344611 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:31:53.346008 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:53.346413 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:53.346416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:53.346417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:53.346494 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:31:53.346889 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:31:53.346892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:31:53.346894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:31:53.347114 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:31:53.347308 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.347575 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.347670 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 12:31:53.350719 22890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:31:53.353670 22890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:31:53.368355 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.361781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40579 (local address 127.0.0.1:53024)
0806 12:31:53.362943 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.362994 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.363291 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.363828 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.363834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.363857 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:53.364009 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.364019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.365525 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.365531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.366616 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.366626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.367791 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.367794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.367901 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.367938 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.368069 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.368198 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"spinlock_wait_cycles":896,"thread_start_us":73,"threads_started":1}
I20260806 12:31:53.368363 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.361913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.6.152.130:40579)
0806 12:31:53.362785 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.362799 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.363229 (+ 430us) server_negotiation.cc:407] Connection header received
0806 12:31:53.363462 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.363468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.363646 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.363825 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:53.364155 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.365373 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.366754 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.367648 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.367763 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.367899 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.368110 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.368213 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":173,"threads_started":1}
I20260806 12:31:53.368700 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.362346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.6.152.129:34393)
0806 12:31:53.362662 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.362679 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.363316 (+ 637us) server_negotiation.cc:407] Connection header received
0806 12:31:53.363432 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.363438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.363646 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.363825 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:53.364474 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.365601 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.367062 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.367687 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.367819 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.367965 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.368424 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.368571 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260806 12:31:53.368355 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.362057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34393 (local address 127.0.0.1:39604)
0806 12:31:53.362993 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.363008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.363181 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.364026 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.364029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.364061 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:53.364268 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.364274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.365754 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.365757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.366936 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.366945 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.368032 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.368035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.368122 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.368141 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.368207 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.368259 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"mutex_wait_us":71,"spinlock_wait_cycles":5248,"thread_start_us":89,"threads_started":1}
I20260806 12:31:53.369699 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.362528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44807 (local address 127.0.0.1:50914)
0806 12:31:53.363426 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.363441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.363517 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.364031 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.364034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.364062 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:53.364258 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.364267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.366895 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.366898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.368004 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.368014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.369312 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.369315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.369409 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.369433 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.369504 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.369568 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":54,"spinlock_wait_cycles":6656,"thread_start_us":92,"threads_started":1}
I20260806 12:31:53.370131 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.362948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.6.152.131:44807)
0806 12:31:53.363577 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.363590 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.363612 (+ 22us) server_negotiation.cc:407] Connection header received
0806 12:31:53.363683 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.363687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.363901 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.364078 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:53.365573 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.366758 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.368173 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.369002 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.369112 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.369247 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.369881 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.369992 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":138,"threads_started":1}
I20260806 12:31:53.371925 23157 tablet_service.cc:1467] Processing CreateTablet for tablet abdfb2964a4c4628af052729d84919aa (DEFAULT_TABLE table=test-table [id=a4dfa11a70264ebd9c4a59fe618c60c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:53.371965 23019 tablet_service.cc:1467] Processing CreateTablet for tablet abdfb2964a4c4628af052729d84919aa (DEFAULT_TABLE table=test-table [id=a4dfa11a70264ebd9c4a59fe618c60c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:53.372946 23157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abdfb2964a4c4628af052729d84919aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:53.373000 23019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abdfb2964a4c4628af052729d84919aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:53.373003 23295 tablet_service.cc:1467] Processing CreateTablet for tablet abdfb2964a4c4628af052729d84919aa (DEFAULT_TABLE table=test-table [id=a4dfa11a70264ebd9c4a59fe618c60c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:31:53.373952 23295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abdfb2964a4c4628af052729d84919aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:31:53.380089 23382 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Bootstrap starting.
I20260806 12:31:53.380991 23383 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap starting.
I20260806 12:31:53.382401 23382 tablet_bootstrap.cc:654] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:53.382685 23383 tablet_bootstrap.cc:654] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:53.382893 23384 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Bootstrap starting.
I20260806 12:31:53.383315 23382 log.cc:824] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:53.383999 23383 log.cc:824] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:53.385231 23384 tablet_bootstrap.cc:654] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Neither blocks nor log segments found. Creating new log.
I20260806 12:31:53.385365 23382 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: No bootstrap required, opened a new log
I20260806 12:31:53.385576 23382 ts_tablet_manager.cc:1397] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:31:53.386161 23384 log.cc:824] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Log is configured to *not* fsync() on all Append() calls
I20260806 12:31:53.386778 23383 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: No bootstrap required, opened a new log
I20260806 12:31:53.387024 23383 ts_tablet_manager.cc:1397] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:31:53.388145 23384 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: No bootstrap required, opened a new log
I20260806 12:31:53.388391 23384 ts_tablet_manager.cc:1397] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:31:53.389147 23382 raft_consensus.cc:348] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.389403 23382 raft_consensus.cc:374] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:53.389456 23382 raft_consensus.cc:729] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419532933013415da09839e93129881e, State: Initialized, Role: FOLLOWER
I20260806 12:31:53.389914 23382 consensus_queue.cc:260] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.390586 23226 heartbeater.cc:503] Master 127.6.152.190:38245 was elected leader, sending a full tablet report...
I20260806 12:31:53.390894 23382 ts_tablet_manager.cc:1428] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:31:53.390991 23383 raft_consensus.cc:348] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.391250 23383 raft_consensus.cc:374] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:53.391316 23383 raft_consensus.cc:729] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736ac31fd2bd4acd9161c033535c0d20, State: Initialized, Role: FOLLOWER
I20260806 12:31:53.391747 23383 consensus_queue.cc:260] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.392621 23383 ts_tablet_manager.cc:1428] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:31:53.392980 23087 heartbeater.cc:503] Master 127.6.152.190:38245 was elected leader, sending a full tablet report...
I20260806 12:31:53.393577 23384 raft_consensus.cc:348] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.393860 23384 raft_consensus.cc:374] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:31:53.393939 23384 raft_consensus.cc:729] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667a68c7828a4fddbcca08c2333b10a5, State: Initialized, Role: FOLLOWER
I20260806 12:31:53.394356 23384 consensus_queue.cc:260] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.395022 23365 heartbeater.cc:503] Master 127.6.152.190:38245 was elected leader, sending a full tablet report...
I20260806 12:31:53.395395 23384 ts_tablet_manager.cc:1428] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 12:31:53.561548 23366 tablet.cc:2369] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:31:53.593011 23227 tablet.cc:2369] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:31:53.618188 23388 raft_consensus.cc:482] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:31:53.618387 23388 raft_consensus.cc:504] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.619693 23388 leader_election.cc:288] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393), 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807)
I20260806 12:31:53.625015 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.620089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:39501 (local address 127.6.152.129:34393)
0806 12:31:53.620243 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.620250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.620766 (+ 516us) server_negotiation.cc:407] Connection header received
0806 12:31:53.620912 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.620915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.620970 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.621082 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:53.621477 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.621996 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.623606 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.624250 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.624410 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.624459 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.624802 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.624881 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 12:31:53.625013 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.620131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:39515 (local address 127.6.152.131:44807)
0806 12:31:53.620329 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.620333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.620373 (+ 40us) server_negotiation.cc:407] Connection header received
0806 12:31:53.620595 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.620598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.620662 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.620769 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:31:53.621297 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.621920 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.623615 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.624282 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.624405 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.624459 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.624802 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.624915 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 12:31:53.625065 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.619949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34393 (local address 127.6.152.130:39501)
0806 12:31:53.620642 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.620659 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.620784 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.621117 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.621122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.621165 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:53.621316 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.621325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.622130 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.622133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.623458 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.623471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.624413 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.624417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.624630 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.624693 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.624769 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.624885 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":373,"thread_start_us":94,"threads_started":1}
I20260806 12:31:53.625068 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.619949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44807 (local address 127.6.152.130:39515)
0806 12:31:53.620239 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.620259 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 12:31:53.620384 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:31:53.620829 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:31:53.620853 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:31:53.620880 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:31:53.621056 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.621111 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.622066 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.622073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.623458 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:31:53.623471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.624413 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.624417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:31:53.624602 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.624687 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 12:31:53.624769 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:31:53.624885 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":128,"thread_start_us":93,"threads_started":1}
I20260806 12:31:53.625900 23316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abdfb2964a4c4628af052729d84919aa" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667a68c7828a4fddbcca08c2333b10a5" is_pre_election: true
I20260806 12:31:53.625903 23039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abdfb2964a4c4628af052729d84919aa" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736ac31fd2bd4acd9161c033535c0d20" is_pre_election: true
I20260806 12:31:53.626197 23039 raft_consensus.cc:2454] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 0.
I20260806 12:31:53.626207 23316 raft_consensus.cc:2454] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 0.
I20260806 12:31:53.626724 23111 leader_election.cc:302] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5, 736ac31fd2bd4acd9161c033535c0d20; no voters:
I20260806 12:31:53.626986 23388 raft_consensus.cc:2781] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:31:53.627084 23388 raft_consensus.cc:482] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:31:53.627152 23388 raft_consensus.cc:3037] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Advancing to term 1
W20260806 12:31:53.627555 23088 tablet.cc:2369] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:31:53.628450 23388 raft_consensus.cc:504] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.628800 23388 leader_election.cc:288] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 election: Requested vote from peers 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393), 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807)
I20260806 12:31:53.629144 23039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abdfb2964a4c4628af052729d84919aa" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736ac31fd2bd4acd9161c033535c0d20"
I20260806 12:31:53.629245 23316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abdfb2964a4c4628af052729d84919aa" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667a68c7828a4fddbcca08c2333b10a5"
I20260806 12:31:53.629249 23039 raft_consensus.cc:3037] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:53.629390 23316 raft_consensus.cc:3037] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:31:53.630342 23039 raft_consensus.cc:2454] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 1.
I20260806 12:31:53.630524 23316 raft_consensus.cc:2454] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 1.
I20260806 12:31:53.630668 23109 leader_election.cc:302] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 419532933013415da09839e93129881e, 736ac31fd2bd4acd9161c033535c0d20; no voters:
I20260806 12:31:53.630856 23388 raft_consensus.cc:2781] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:31:53.631095 23388 raft_consensus.cc:686] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 LEADER]: Becoming Leader. State: Replica: 419532933013415da09839e93129881e, State: Running, Role: LEADER
I20260806 12:31:53.631460 23388 consensus_queue.cc:237] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:31:53.634584 22890 catalog_manager.cc:5495] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e reported cstate change: term changed from 0 to 1, leader changed from <none> to 419532933013415da09839e93129881e (127.6.152.130). New cstate: current_term: 1 leader_uuid: "419532933013415da09839e93129881e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } health_report { overall_health: UNKNOWN } } }
I20260806 12:31:53.658689 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.655080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.6.152.130:40579)
0806 12:31:53.655233 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 12:31:53.655239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:31:53.655253 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:31:53.655447 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:31:53.655450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:31:53.655506 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:31:53.655620 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:31:53.655867 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.656361 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.657254 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:31:53.657473 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:31:53.657581 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:31:53.657905 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:31:53.658297 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:31:53.658437 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 12:31:53.658473 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 12:31:53.658522 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:31:53.664529 23316 raft_consensus.cc:1261] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Refusing update from remote peer 419532933013415da09839e93129881e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:53.664538 23039 raft_consensus.cc:1261] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Refusing update from remote peer 419532933013415da09839e93129881e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:31:53.665283 23394 consensus_queue.cc:1035] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [LEADER]: Connected to new peer: Peer: permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:53.665457 23388 consensus_queue.cc:1035] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [LEADER]: Connected to new peer: Peer: permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:31:53.672483 23397 mvcc.cc:205] Tried to move back new op lower bound from 7315535927963090944 to 7315535927836909568. Current Snapshot: MvccSnapshot[applied={T|T < 7315535927963090944}]
I20260806 12:31:53.673137 23398 mvcc.cc:205] Tried to move back new op lower bound from 7315535927963090944 to 7315535927836909568. Current Snapshot: MvccSnapshot[applied={T|T < 7315535927963090944}]
I20260806 12:31:53.674037 23399 mvcc.cc:205] Tried to move back new op lower bound from 7315535927963090944 to 7315535927836909568. Current Snapshot: MvccSnapshot[applied={T|T < 7315535927963090944}]
I20260806 12:31:53.680328 22888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:53.680516 22888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:31:53.682788 22888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Sending DeleteTablet for 3 replicas of tablet abdfb2964a4c4628af052729d84919aa
I20260806 12:31:53.683558 23157 tablet_service.cc:1514] Processing DeleteTablet for tablet abdfb2964a4c4628af052729d84919aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:53024
I20260806 12:31:53.683635 23295 tablet_service.cc:1514] Processing DeleteTablet for tablet abdfb2964a4c4628af052729d84919aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:50914
I20260806 12:31:53.684255 23019 tablet_service.cc:1514] Processing DeleteTablet for tablet abdfb2964a4c4628af052729d84919aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:39604
I20260806 12:31:53.684315 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 22955
I20260806 12:31:53.687335 23407 tablet_replica.cc:330] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: stopping tablet replica
I20260806 12:31:53.687716 23407 raft_consensus.cc:2229] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:31:53.688056 23407 raft_consensus.cc:2258] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:53.689705 23407 ts_tablet_manager.cc:1905] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:31:53.690011 23406 tablet_replica.cc:330] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: stopping tablet replica
I20260806 12:31:53.690299 23406 raft_consensus.cc:2229] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:31:53.690660 23406 raft_consensus.cc:2258] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:31:53.692044 23406 ts_tablet_manager.cc:1905] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 12:31:53.693166 22875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:34393 (error 108)
I20260806 12:31:53.693413 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 23092
I20260806 12:31:53.695245 23407 ts_tablet_manager.cc:1918] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:31:53.695391 23407 log.cc:1196] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/abdfb2964a4c4628af052729d84919aa
I20260806 12:31:53.695740 23407 ts_tablet_manager.cc:1939] T abdfb2964a4c4628af052729d84919aa P 667a68c7828a4fddbcca08c2333b10a5: Deleting consensus metadata
I20260806 12:31:53.696523 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.696201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34393 (local address 127.0.0.1:39612)
0806 12:31:53.696358 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.696437 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:34393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 12:31:53.696775 22875 catalog_manager.cc:4584] TS 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet abdfb2964a4c4628af052729d84919aa: Network error: Client connection negotiation failed: client connection to 127.6.152.129:34393: connect: Connection refused (error 111)
I20260806 12:31:53.697121 22877 catalog_manager.cc:4841] TS 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): tablet abdfb2964a4c4628af052729d84919aa (table test-table [id=a4dfa11a70264ebd9c4a59fe618c60c9]) successfully deleted
I20260806 12:31:53.701310 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 23231
I20260806 12:31:53.702165 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:31:53.701485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40579 (local address 127.0.0.1:53042)
0806 12:31:53.701994 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 12:31:53.702070 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.130:40579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421}
I20260806 12:31:53.708261 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 22858
I20260806 12:32:13.716665 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38245
--webserver_interface=127.6.152.190
--webserver_port=40749
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:38245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:13.831984 23409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:13.832281 23409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:13.832360 23409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:13.836308 23409 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 12:32:13.836391 23409 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 12:32:13.836423 23409 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 12:32:13.836454 23409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:13.836488 23409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:13.836532 23409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:13.836565 23409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:13.836602 23409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:13.841987 23409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:38245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=40749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:13.843040 23409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:13.844079 23409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:13.857262 23415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:13.857306 23418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:13.857445 23409 server_base.cc:1034] running on GCE node
W20260806 12:32:13.857271 23416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:13.858150 23409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:13.859387 23409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:13.860600 23409 hybrid_clock.cc:648] HybridClock initialized: now 1786019533860570 us; error 42 us; skew 500 ppm
Aug 06 12:32:13 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:13.863409 23409 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:32:13.864305 23409 webserver.cc:460] Webserver started at http://127.6.152.190:40749/ using document root <none> and password file <none>
I20260806 12:32:13.864893 23409 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:13.865002 23409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:13.868896 23409 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 12:32:13.871013 23425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:13.872213 23409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:32:13.872370 23409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "d925806e76ef4db29da179ede05c9a91"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "6eb1b8cdad2e3614313982ee772dacb0"
server_key_iv: "6fbd66cbf2ae26c202a455b21b413c7f"
server_key_version: "encryptionkey@0"
I20260806 12:32:13.872973 23409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:13.883865 23409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:13.886650 23409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:13.886878 23409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:13.893833 23409 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:38245
I20260806 12:32:13.894121 23478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:38245 every 8 connection(s)
I20260806 12:32:13.894904 23409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:13.899909 23479 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:32:13.901515 23479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap starting.
I20260806 12:32:13.904045 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23409
I20260806 12:32:13.904490 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:34393
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:38245
--webserver_port=38813
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:32:13.920581 23479 log.cc:824] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:13.929128 23479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:32:13.929514 23479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap complete.
I20260806 12:32:13.933621 23479 raft_consensus.cc:348] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:13.934007 23479 raft_consensus.cc:729] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d925806e76ef4db29da179ede05c9a91, State: Initialized, Role: FOLLOWER
I20260806 12:32:13.934479 23479 consensus_queue.cc:260] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:13.934588 23479 raft_consensus.cc:388] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:13.934679 23479 raft_consensus.cc:482] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:13.934793 23479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 1 FOLLOWER]: Advancing to term 2
I20260806 12:32:13.936687 23479 raft_consensus.cc:504] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:13.937027 23479 leader_election.cc:302] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d925806e76ef4db29da179ede05c9a91; no voters:
I20260806 12:32:13.937345 23479 leader_election.cc:288] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 12:32:13.937428 23483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 12:32:13.937618 23483 raft_consensus.cc:686] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 LEADER]: Becoming Leader. State: Replica: d925806e76ef4db29da179ede05c9a91, State: Running, Role: LEADER
I20260806 12:32:13.937907 23483 consensus_queue.cc:237] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:13.938664 23479 sys_catalog.cc:564] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:13.938810 23483 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d925806e76ef4db29da179ede05c9a91. Latest consensus state: current_term: 2 leader_uuid: "d925806e76ef4db29da179ede05c9a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } } }
I20260806 12:32:13.938923 23483 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:13.939425 23484 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d925806e76ef4db29da179ede05c9a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } } }
I20260806 12:32:13.939718 23484 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:13.939430 23487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:13.944869 23487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:13.945479 23487 catalog_manager.cc:1259] Loaded cluster ID: 73b993b7b35645969bcb60968cf53d43
I20260806 12:32:13.945540 23487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:13.947988 23487 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:13.948693 23487 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Loaded TSK: 0
I20260806 12:32:13.966802 23487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Generated new TSK 1
I20260806 12:32:13.967070 23487 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:32:14.020345 23481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:14.020689 23481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:14.020771 23481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:14.024819 23481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:14.024899 23481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:14.024986 23481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:14.030025 23481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:34393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=38813
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:14.031181 23481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:14.032172 23481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:14.043277 23511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:14.043303 23508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:14.043587 23509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:14.043712 23481 server_base.cc:1034] running on GCE node
I20260806 12:32:14.044162 23481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:14.044879 23481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:14.046089 23481 hybrid_clock.cc:648] HybridClock initialized: now 1786019534046053 us; error 40 us; skew 500 ppm
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:14.049394 23481 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:32:14.050295 23481 webserver.cc:460] Webserver started at http://127.6.152.129:38813/ using document root <none> and password file <none>
I20260806 12:32:14.050932 23481 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:14.051044 23481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:14.055212 23481 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:32:14.057426 23518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:14.058696 23481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:14.058830 23481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "736ac31fd2bd4acd9161c033535c0d20"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "89a1933c9a14631daf847ff0ede27f31"
server_key_iv: "65f0f2efd502949dd46fa4f6c0849946"
server_key_version: "encryptionkey@0"
I20260806 12:32:14.059409 23481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:14.073378 23481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:14.076629 23481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:14.076936 23481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:14.077545 23481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:14.078702 23525 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:32:14.082777 23481 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:32:14.082841 23481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260806 12:32:14.082887 23481 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:32:14.084612 23481 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:32:14.084671 23481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 12:32:14.084919 23525 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap starting.
I20260806 12:32:14.096282 23481 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:34393
I20260806 12:32:14.096339 23636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:34393 every 8 connection(s)
I20260806 12:32:14.097479 23481 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:14.101848 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23481
I20260806 12:32:14.102368 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:40579
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:38245
--webserver_port=40493
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:32:14.107586 23525 log.cc:824] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:14.116499 23525 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:32:14.116946 23525 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap complete.
I20260806 12:32:14.117725 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.099725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:45837)
0806 12:32:14.100324 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.100366 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.101236 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.102780 (+ 1544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.102789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.103319 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:14.104127 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.104150 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.105972 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.105976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.107303 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.107315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.108435 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.108443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.108621 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.109297 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:14.109317 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:14.111419 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:14.114160 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.114166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.114181 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.114563 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.114997 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.115002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.115005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.115237 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.115932 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:14.115938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:14.116219 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.116399 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.116708 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":138,"threads_started":1}
I20260806 12:32:14.118515 23525 ts_tablet_manager.cc:1397] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.008s
I20260806 12:32:14.118569 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.099687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:45837 (local address 127.6.152.190:38245)
0806 12:32:14.100080 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.100094 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.100514 (+ 420us) server_negotiation.cc:407] Connection header received
0806 12:32:14.101676 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.101692 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.102103 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.102474 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.104347 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.105796 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.107433 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.108260 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.108459 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.111652 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.111658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.111674 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.111707 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.114002 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.114717 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.114723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.114728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.114876 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.115353 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.115357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.115358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.115804 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.115945 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.116677 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.117266 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":95,"threads_started":1}
I20260806 12:32:14.120914 23637 heartbeater.cc:347] Connected to a master server at 127.6.152.190:38245
I20260806 12:32:14.121142 23637 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:14.121188 23525 raft_consensus.cc:348] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:32:14.121680 23525 raft_consensus.cc:729] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736ac31fd2bd4acd9161c033535c0d20, State: Initialized, Role: FOLLOWER
I20260806 12:32:14.121688 23637 heartbeater.cc:511] Master 127.6.152.190:38245 requested a full tablet report, sending...
I20260806 12:32:14.122152 23525 consensus_queue.cc:260] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:32:14.123195 23525 ts_tablet_manager.cc:1428] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260806 12:32:14.124753 23439 ts_manager.cc:194] Registered new tserver with Master: 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393)
I20260806 12:32:14.128619 23439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:45837
I20260806 12:32:14.131091 23637 heartbeater.cc:503] Master 127.6.152.190:38245 was elected leader, sending a full tablet report...
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.129@KRBTEST.COM
I20260806 12:32:14.138748 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.128006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34393 (local address 127.0.0.1:43698)
0806 12:32:14.128692 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.128762 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.128912 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.129374 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.129379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.129780 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:14.129913 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.129920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.131235 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.131239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.132140 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.132152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.132860 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.132864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.132980 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.133491 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:14.133506 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:14.134996 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:14.136930 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.136942 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.136947 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.137189 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.137515 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.137519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.137521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.137625 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.138129 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:14.138136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:14.138255 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.138402 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.138583 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":117,"threads_started":1}
I20260806 12:32:14.139252 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.128055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.6.152.129:34393)
0806 12:32:14.128433 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.128446 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.128843 (+ 397us) server_negotiation.cc:407] Connection header received
0806 12:32:14.129068 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.129078 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.129225 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.129378 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.130068 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.131069 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.132283 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.132742 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.132828 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.135165 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.135180 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.135189 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.135211 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.136802 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.137330 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.137335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.137336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.137414 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.137726 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.137732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.137733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.137973 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.138154 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.138507 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.138713 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":97,"threads_started":1}
I20260806 12:32:14.140199 23570 tablet_service.cc:1514] Processing DeleteTablet for tablet abdfb2964a4c4628af052729d84919aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:43698
I20260806 12:32:14.141233 23647 tablet_replica.cc:330] stopping tablet replica
I20260806 12:32:14.141511 23647 raft_consensus.cc:2229] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:14.141749 23647 raft_consensus.cc:2258] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:14.142657 23647 ts_tablet_manager.cc:1905] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:14.146237 23647 ts_tablet_manager.cc:1918] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:14.146353 23647 log.cc:1196] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal/wals/abdfb2964a4c4628af052729d84919aa
I20260806 12:32:14.146626 23647 ts_tablet_manager.cc:1939] T abdfb2964a4c4628af052729d84919aa P 736ac31fd2bd4acd9161c033535c0d20: Deleting consensus metadata
I20260806 12:32:14.148301 23427 catalog_manager.cc:4841] TS 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393): tablet abdfb2964a4c4628af052729d84919aa (table test-table [id=a4dfa11a70264ebd9c4a59fe618c60c9]) successfully deleted
W20260806 12:32:14.221309 23642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:14.221560 23642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:14.221601 23642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:14.225608 23642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:14.225659 23642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:14.225752 23642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:14.230791 23642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:40579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=40493
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:14.231783 23642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:14.232745 23642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:32:14.242198 23642 server_base.cc:1034] running on GCE node
W20260806 12:32:14.242200 23652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:14.242201 23655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:14.242437 23653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:14.242844 23642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:14.243551 23642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:14.244740 23642 hybrid_clock.cc:648] HybridClock initialized: now 1786019534244711 us; error 43 us; skew 500 ppm
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:14.247459 23642 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:32:14.248291 23642 webserver.cc:460] Webserver started at http://127.6.152.130:40493/ using document root <none> and password file <none>
I20260806 12:32:14.248853 23642 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:14.248966 23642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:14.252688 23642 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 12:32:14.254915 23662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:14.256139 23642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:14.256270 23642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "419532933013415da09839e93129881e"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "ec57b580915b93a2682aa3c3d19c3bd5"
server_key_iv: "36d2f5f3dbf62a3e0fff70cd82e9ff5c"
server_key_version: "encryptionkey@0"
I20260806 12:32:14.256808 23642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:14.278448 23642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:14.281508 23642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:14.281724 23642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:14.282317 23642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:14.283406 23670 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:32:14.288662 23642 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:32:14.288762 23642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 12:32:14.288843 23642 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:32:14.290499 23642 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:32:14.290575 23642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 12:32:14.290812 23670 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Bootstrap starting.
I20260806 12:32:14.296176 23670 log.cc:824] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:14.301051 23642 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:40579
I20260806 12:32:14.301170 23782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:40579 every 8 connection(s)
I20260806 12:32:14.301905 23642 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:32:14.304200 23670 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:32:14.304620 23670 tablet_bootstrap.cc:492] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Bootstrap complete.
I20260806 12:32:14.304955 23670 ts_tablet_manager.cc:1397] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260806 12:32:14.307811 23670 raft_consensus.cc:348] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:32:14.308285 23670 raft_consensus.cc:729] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419532933013415da09839e93129881e, State: Initialized, Role: FOLLOWER
I20260806 12:32:14.308748 23670 consensus_queue.cc:260] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } }
I20260806 12:32:14.309674 23670 ts_tablet_manager.cc:1428] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:32:14.310485 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23642
I20260806 12:32:14.310930 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:44807
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:38245
--webserver_port=34403
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:14.316877 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.303743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:59731 (local address 127.6.152.190:38245)
0806 12:32:14.303908 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.303913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.304552 (+ 639us) server_negotiation.cc:407] Connection header received
0806 12:32:14.305685 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.305688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.305734 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.305808 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.307214 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.307735 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.309297 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.309519 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.309613 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.312783 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.312789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.312794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.312829 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.314625 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.315106 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.315113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.315117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.315243 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.315598 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.315604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.315609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.315875 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.316003 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.316465 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.316638 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:32:14.317600 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.303871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:59731)
0806 12:32:14.304377 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.304434 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.305473 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.305988 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.305997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.306448 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:14.307058 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.307075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.307897 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.307901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.309165 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.309175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.309651 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.309655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.309885 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.310953 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:14.310969 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:14.312555 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:14.314740 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.314744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.314756 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.315002 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.315342 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.315344 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.315345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.315494 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.316000 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:14.316005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:14.316223 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.316375 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.316630 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":123,"threads_started":1}
I20260806 12:32:14.318822 23783 heartbeater.cc:347] Connected to a master server at 127.6.152.190:38245
I20260806 12:32:14.319159 23783 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:14.319775 23783 heartbeater.cc:511] Master 127.6.152.190:38245 requested a full tablet report, sending...
I20260806 12:32:14.321861 23439 ts_manager.cc:194] Registered new tserver with Master: 419532933013415da09839e93129881e (127.6.152.130:40579)
I20260806 12:32:14.323012 23439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:59731
I20260806 12:32:14.325436 23783 heartbeater.cc:503] Master 127.6.152.190:38245 was elected leader, sending a full tablet report...
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.130@KRBTEST.COM
I20260806 12:32:14.332103 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.322771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40579 (local address 127.0.0.1:47414)
0806 12:32:14.322894 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.322908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.322996 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.323906 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.323909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.324119 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:14.324198 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.324204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.325169 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.325175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.325836 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.325849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.326498 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.326501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.326598 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.327274 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:14.327286 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:14.328552 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:14.330585 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.330588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.330589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.330775 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.331107 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.331109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.331110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.331248 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.331711 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:14.331713 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:14.331783 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.331865 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.331992 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 12:32:14.332312 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.323205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.6.152.130:40579)
0806 12:32:14.323531 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.323541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.323558 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:32:14.323611 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.323615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.323767 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.323923 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.324314 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.325046 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.326029 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.326371 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.326495 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.328683 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.328693 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.328707 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.328732 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.330481 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.330886 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.330894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.330898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.331001 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.331336 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.331339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.331341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.331611 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.331741 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.331982 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.332145 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":107,"threads_started":1}
I20260806 12:32:14.333325 23713 tablet_service.cc:1514] Processing DeleteTablet for tablet abdfb2964a4c4628af052729d84919aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:31:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:47414
I20260806 12:32:14.334415 23789 tablet_replica.cc:330] stopping tablet replica
I20260806 12:32:14.334745 23789 raft_consensus.cc:2229] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:14.335016 23789 raft_consensus.cc:2258] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:14.336043 23789 ts_tablet_manager.cc:1905] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:14.340010 23789 ts_tablet_manager.cc:1918] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:14.340129 23789 log.cc:1196] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/abdfb2964a4c4628af052729d84919aa
I20260806 12:32:14.340407 23789 ts_tablet_manager.cc:1939] T abdfb2964a4c4628af052729d84919aa P 419532933013415da09839e93129881e: Deleting consensus metadata
I20260806 12:32:14.341971 23426 catalog_manager.cc:4841] TS 419532933013415da09839e93129881e (127.6.152.130:40579): tablet abdfb2964a4c4628af052729d84919aa (table test-table [id=a4dfa11a70264ebd9c4a59fe618c60c9]) successfully deleted
W20260806 12:32:14.429129 23787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:14.429368 23787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:14.429410 23787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:14.433324 23787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:14.433374 23787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:14.433431 23787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:32:14.438567 23787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:44807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=34403
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:14.439651 23787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:14.440687 23787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:14.450065 23795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:14.450060 23794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:14.450058 23797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:14.450205 23787 server_base.cc:1034] running on GCE node
I20260806 12:32:14.450654 23787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:14.451403 23787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:14.452569 23787 hybrid_clock.cc:648] HybridClock initialized: now 1786019534452551 us; error 28 us; skew 500 ppm
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:14.455381 23787 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:32:14.456189 23787 webserver.cc:460] Webserver started at http://127.6.152.131:34403/ using document root <none> and password file <none>
I20260806 12:32:14.456725 23787 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:14.456825 23787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:14.460639 23787 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:32:14.462674 23804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:14.463992 23787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:14.464084 23787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "667a68c7828a4fddbcca08c2333b10a5"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "3fdd83035532e1a1aa6ef4194bbed3d9"
server_key_iv: "debd4fb719ee576b225d7e99fb685808"
server_key_version: "encryptionkey@0"
I20260806 12:32:14.464576 23787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:14.477537 23787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:14.480234 23787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:14.480378 23787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:14.480870 23787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:14.481588 23787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:14.481634 23787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:14.481673 23787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:14.481694 23787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:14.490232 23787 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:44807
I20260806 12:32:14.490247 23923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:44807 every 8 connection(s)
I20260806 12:32:14.491246 23787 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:32:14.498615 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23787
Aug 06 12:32:14 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:14.504724 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.493032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:45197 (local address 127.6.152.190:38245)
0806 12:32:14.493199 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.493204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.494188 (+ 984us) server_negotiation.cc:407] Connection header received
0806 12:32:14.494977 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.494980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.495025 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.495094 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.496247 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.496766 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.497789 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.497988 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.498059 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.500970 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.500974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.500976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.501005 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.502401 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.502976 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.502982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.502987 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.503075 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.503500 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.503507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.503511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.503723 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.503811 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.504443 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.504532 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 12:32:14.505513 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.493413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:45197)
0806 12:32:14.494046 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.494085 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.494796 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.495314 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.495321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.495672 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:14.496119 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.496131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.496869 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.496871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.497685 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.497694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.498110 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.498112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.498257 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.498855 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:14.498870 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:14.500808 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:14.502529 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.502536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.502549 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.502883 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.503199 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:14.503202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:14.503204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:14.503395 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:14.503845 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:14.503852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:14.504176 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.504396 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.504704 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":97,"threads_started":1}
I20260806 12:32:14.506757 23925 heartbeater.cc:347] Connected to a master server at 127.6.152.190:38245
I20260806 12:32:14.506991 23925 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:14.507524 23925 heartbeater.cc:511] Master 127.6.152.190:38245 requested a full tablet report, sending...
I20260806 12:32:14.508154 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.499324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.6.152.190:38245)
0806 12:32:14.499551 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.499556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.499754 (+ 198us) server_negotiation.cc:407] Connection header received
0806 12:32:14.499880 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.499882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.499919 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.500002 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.500955 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.501732 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.502694 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.502956 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.503056 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.504623 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.504627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.504629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.504650 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.506452 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.506895 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.506899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.506904 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.506992 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.507327 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.507329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.507330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.507492 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.507673 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.507865 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.507972 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260806 12:32:14.508813 23441 ts_manager.cc:194] Registered new tserver with Master: 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807)
I20260806 12:32:14.509387 23441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:45197
I20260806 12:32:14.509802 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:32:14.513481 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.510431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.6.152.190:38245)
0806 12:32:14.510533 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.510537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.510706 (+ 169us) server_negotiation.cc:407] Connection header received
0806 12:32:14.510796 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.510799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.510891 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.510990 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:32:14.511243 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.511782 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.512599 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.512803 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.512877 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.513119 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:32:14.513225 (+ 106us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 12:32:14.513399 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50024: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 12:32:14.513558 23928 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50024: token expired
W20260806 12:32:14.513929 23371 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.152.190:38245)
I20260806 12:32:14.520484 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.514400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.6.152.190:38245)
0806 12:32:14.514530 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.514533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.514545 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:32:14.514573 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.514575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.514605 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.514673 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:14.515251 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.515708 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.516445 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.516636 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.516709 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.517980 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:14.517983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:14.517985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:14.518005 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:14.519229 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.519591 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.519593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.519594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.519663 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:14.519887 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:14.519889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:14.519890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:14.520036 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:14.520130 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.520300 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.520384 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:32:14.522851 23441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:32:14.526317 23441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:32:14.533905 23713 tablet_service.cc:1467] Processing CreateTablet for tablet 92a1fa3f8d5d43e293c9112227976e62 (DEFAULT_TABLE table=test-table [id=2e78ee78676a4c2ca7f8cdcc7288030b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:14.533905 23570 tablet_service.cc:1467] Processing CreateTablet for tablet 92a1fa3f8d5d43e293c9112227976e62 (DEFAULT_TABLE table=test-table [id=2e78ee78676a4c2ca7f8cdcc7288030b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:14.534533 23570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92a1fa3f8d5d43e293c9112227976e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:14.534533 23713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92a1fa3f8d5d43e293c9112227976e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:14.537505 23525 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap starting.
I20260806 12:32:14.537981 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.533208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44807 (local address 127.0.0.1:34572)
0806 12:32:14.533373 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.533388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.533467 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.534191 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.534194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.534211 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:14.534313 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.534321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.535885 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.535892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.536662 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.536671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.537572 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.537576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.537669 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.537721 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.537779 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.537855 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 12:32:14.538439 23525 tablet_bootstrap.cc:654] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:14.538666 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.533374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.6.152.131:44807)
0806 12:32:14.533728 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.533739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.533767 (+ 28us) server_negotiation.cc:407] Connection header received
0806 12:32:14.533831 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.533835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.533996 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.534182 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:14.534449 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.535765 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.536800 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.537420 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.538173 (+ 753us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.538327 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.538385 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.538487 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":173,"threads_started":1}
I20260806 12:32:14.539172 23670 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Bootstrap starting.
I20260806 12:32:14.540211 23670 tablet_bootstrap.cc:654] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:14.541633 23525 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: No bootstrap required, opened a new log
I20260806 12:32:14.541730 23525 ts_tablet_manager.cc:1397] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 12:32:14.542207 23525 raft_consensus.cc:348] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.542325 23525 raft_consensus.cc:374] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:14.542361 23525 raft_consensus.cc:729] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736ac31fd2bd4acd9161c033535c0d20, State: Initialized, Role: FOLLOWER
I20260806 12:32:14.542003 23855 tablet_service.cc:1467] Processing CreateTablet for tablet 92a1fa3f8d5d43e293c9112227976e62 (DEFAULT_TABLE table=test-table [id=2e78ee78676a4c2ca7f8cdcc7288030b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:14.542496 23525 consensus_queue.cc:260] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.543088 23670 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: No bootstrap required, opened a new log
I20260806 12:32:14.543177 23855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92a1fa3f8d5d43e293c9112227976e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:14.543221 23670 ts_tablet_manager.cc:1397] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 12:32:14.543370 23525 ts_tablet_manager.cc:1428] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 12:32:14.543655 23670 raft_consensus.cc:348] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.543773 23670 raft_consensus.cc:374] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:14.543825 23670 raft_consensus.cc:729] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419532933013415da09839e93129881e, State: Initialized, Role: FOLLOWER
I20260806 12:32:14.543962 23670 consensus_queue.cc:260] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.544240 23670 ts_tablet_manager.cc:1428] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 12:32:14.550926 23938 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Bootstrap starting.
W20260806 12:32:14.552775 23784 tablet.cc:2369] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:14.553200 23938 tablet_bootstrap.cc:654] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:14.553930 23938 log.cc:824] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:14.555863 23938 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: No bootstrap required, opened a new log
I20260806 12:32:14.556075 23938 ts_tablet_manager.cc:1397] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:32:14.558760 23938 raft_consensus.cc:348] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.559013 23938 raft_consensus.cc:374] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:14.559073 23938 raft_consensus.cc:729] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667a68c7828a4fddbcca08c2333b10a5, State: Initialized, Role: FOLLOWER
I20260806 12:32:14.559470 23938 consensus_queue.cc:260] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.560159 23925 heartbeater.cc:503] Master 127.6.152.190:38245 was elected leader, sending a full tablet report...
I20260806 12:32:14.560539 23938 ts_tablet_manager.cc:1428] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
W20260806 12:32:14.598856 23639 tablet.cc:2369] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:14.606654 23786 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:14.606876 23786 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.608017 23786 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393)
I20260806 12:32:14.611925 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.608360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:47321 (local address 127.6.152.131:44807)
0806 12:32:14.608490 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.608497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.608518 (+ 21us) server_negotiation.cc:407] Connection header received
0806 12:32:14.608752 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.608755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.608800 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.608889 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:14.609175 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.609697 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.610833 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.611329 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.611466 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.611521 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.611736 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.611803 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 12:32:14.611925 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.608254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44807 (local address 127.6.152.130:47321)
0806 12:32:14.608462 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.608479 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.608590 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.608904 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.608907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.608925 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:14.609055 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.609061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.609821 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.609826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.610701 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.610710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.611458 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.611462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.611574 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.611622 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.611718 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.611792 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1}
I20260806 12:32:14.612010 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.608404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:34935 (local address 127.6.152.129:34393)
0806 12:32:14.608545 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.608551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.608756 (+ 205us) server_negotiation.cc:407] Connection header received
0806 12:32:14.608908 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.608912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.608970 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.609066 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:14.609355 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.609894 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.610993 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.611489 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.611627 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.611774 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.611820 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.611882 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 12:32:14.612116 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.608274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34393 (local address 127.6.152.130:34935)
0806 12:32:14.608657 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.608667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:14.608792 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:14.609091 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:14.609094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:14.609111 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:14.609239 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.609244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.610051 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.610054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.610846 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:14.610854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.611628 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.611632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:14.611721 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.611744 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:32:14.611810 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 12:32:14.611876 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":23,"thread_start_us":119,"threads_started":1}
I20260806 12:32:14.612636 23877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a1fa3f8d5d43e293c9112227976e62" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667a68c7828a4fddbcca08c2333b10a5" is_pre_election: true
I20260806 12:32:14.612664 23590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a1fa3f8d5d43e293c9112227976e62" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736ac31fd2bd4acd9161c033535c0d20" is_pre_election: true
I20260806 12:32:14.612931 23877 raft_consensus.cc:2454] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 0.
I20260806 12:32:14.612980 23590 raft_consensus.cc:2454] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 0.
I20260806 12:32:14.613457 23666 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5, 736ac31fd2bd4acd9161c033535c0d20; no voters:
I20260806 12:32:14.613736 23786 raft_consensus.cc:2781] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:32:14.613808 23786 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:32:14.613839 23786 raft_consensus.cc:3037] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:14.615100 23786 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.615494 23786 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 election: Requested vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393)
I20260806 12:32:14.615809 23590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a1fa3f8d5d43e293c9112227976e62" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736ac31fd2bd4acd9161c033535c0d20"
I20260806 12:32:14.615823 23877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a1fa3f8d5d43e293c9112227976e62" candidate_uuid: "419532933013415da09839e93129881e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667a68c7828a4fddbcca08c2333b10a5"
I20260806 12:32:14.615954 23590 raft_consensus.cc:3037] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:14.615970 23877 raft_consensus.cc:3037] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:14.616969 23877 raft_consensus.cc:2454] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 1.
I20260806 12:32:14.616990 23590 raft_consensus.cc:2454] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 419532933013415da09839e93129881e in term 1.
I20260806 12:32:14.617313 23664 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 419532933013415da09839e93129881e, 736ac31fd2bd4acd9161c033535c0d20; no voters:
I20260806 12:32:14.617522 23786 raft_consensus.cc:2781] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:14.617715 23786 raft_consensus.cc:686] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 1 LEADER]: Becoming Leader. State: Replica: 419532933013415da09839e93129881e, State: Running, Role: LEADER
I20260806 12:32:14.617980 23786 consensus_queue.cc:237] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:14.620818 23439 catalog_manager.cc:5495] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e reported cstate change: term changed from 0 to 1, leader changed from <none> to 419532933013415da09839e93129881e (127.6.152.130). New cstate: current_term: 1 leader_uuid: "419532933013415da09839e93129881e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } health_report { overall_health: HEALTHY } } }
I20260806 12:32:14.641811 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.637773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.6.152.130:40579)
0806 12:32:14.637907 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:14.637914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:14.637998 (+ 84us) server_negotiation.cc:407] Connection header received
0806 12:32:14.638236 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:14.638239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:14.638351 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:14.638440 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:32:14.638769 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.639479 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.640448 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:14.640673 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:14.640752 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:14.640977 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:32:14.641415 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:32:14.641503 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:14.641577 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 12:32:14.641624 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:32:14.645597 23877 raft_consensus.cc:1261] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Refusing update from remote peer 419532933013415da09839e93129881e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:14.645610 23590 raft_consensus.cc:1261] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Refusing update from remote peer 419532933013415da09839e93129881e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:14.646351 23786 consensus_queue.cc:1035] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [LEADER]: Connected to new peer: Peer: permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:14.646556 23942 consensus_queue.cc:1035] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [LEADER]: Connected to new peer: Peer: permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:14.653918 23945 mvcc.cc:205] Tried to move back new op lower bound from 7315536013902635008 to 7315536013797908480. Current Snapshot: MvccSnapshot[applied={T|T < 7315536013902635008}]
I20260806 12:32:14.655189 23944 mvcc.cc:205] Tried to move back new op lower bound from 7315536013902635008 to 7315536013797908480. Current Snapshot: MvccSnapshot[applied={T|T < 7315536013902635008 or (T in {7315536013902635008})}]
I20260806 12:32:14.655393 23946 mvcc.cc:205] Tried to move back new op lower bound from 7315536013902635008 to 7315536013797908480. Current Snapshot: MvccSnapshot[applied={T|T < 7315536013902635008}]
I20260806 12:32:14.660912 23439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:14.661091 23439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:14.663579 23439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Sending DeleteTablet for 3 replicas of tablet 92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:14.664227 23570 tablet_service.cc:1514] Processing DeleteTablet for tablet 92a1fa3f8d5d43e293c9112227976e62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:14 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:43698
I20260806 12:32:14.664266 23855 tablet_service.cc:1514] Processing DeleteTablet for tablet 92a1fa3f8d5d43e293c9112227976e62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:14 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:34572
I20260806 12:32:14.664482 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 23481
I20260806 12:32:14.666355 23713 tablet_service.cc:1514] Processing DeleteTablet for tablet 92a1fa3f8d5d43e293c9112227976e62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:14 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:47414
I20260806 12:32:14.666558 23789 tablet_replica.cc:330] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: stopping tablet replica
I20260806 12:32:14.666698 23789 raft_consensus.cc:2229] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:32:14.666724 23955 tablet_replica.cc:330] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: stopping tablet replica
I20260806 12:32:14.667002 23789 raft_consensus.cc:2258] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:14.667233 23955 raft_consensus.cc:2229] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:14.667523 23955 raft_consensus.cc:2258] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:14.668884 23955 ts_tablet_manager.cc:1905] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:14.672070 23789 ts_tablet_manager.cc:1905] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:14.673978 23955 ts_tablet_manager.cc:1918] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:14.674142 23955 log.cc:1196] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:14.674176 23789 ts_tablet_manager.cc:1918] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:14.674244 23789 log.cc:1196] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:14.674485 23789 ts_tablet_manager.cc:1939] T 92a1fa3f8d5d43e293c9112227976e62 P 419532933013415da09839e93129881e: Deleting consensus metadata
I20260806 12:32:14.674499 23955 ts_tablet_manager.cc:1939] T 92a1fa3f8d5d43e293c9112227976e62 P 667a68c7828a4fddbcca08c2333b10a5: Deleting consensus metadata
I20260806 12:32:14.675246 23426 catalog_manager.cc:4841] TS 419532933013415da09839e93129881e (127.6.152.130:40579): tablet 92a1fa3f8d5d43e293c9112227976e62 (table test-table [id=2e78ee78676a4c2ca7f8cdcc7288030b]) successfully deleted
I20260806 12:32:14.675709 23429 catalog_manager.cc:4841] TS 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): tablet 92a1fa3f8d5d43e293c9112227976e62 (table test-table [id=2e78ee78676a4c2ca7f8cdcc7288030b]) successfully deleted
W20260806 12:32:14.678136 23427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:34393 (error 108)
I20260806 12:32:14.678411 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 23642
I20260806 12:32:14.680457 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:14.680025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:34393 (local address 127.0.0.1:43708)
0806 12:32:14.680251 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:14.680349 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:34393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260806 12:32:14.680652 23427 catalog_manager.cc:4584] TS 736ac31fd2bd4acd9161c033535c0d20 (127.6.152.129:34393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92a1fa3f8d5d43e293c9112227976e62: Network error: Client connection negotiation failed: client connection to 127.6.152.129:34393: connect: Connection refused (error 111)
I20260806 12:32:14.687470 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 23787
I20260806 12:32:14.694414 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 23409
I20260806 12:32:34.702147 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38245
--webserver_interface=127.6.152.190
--webserver_port=40749
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:38245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:34.811209 23957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:34.811517 23957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:34.811591 23957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:34.815558 23957 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 12:32:34.815600 23957 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 12:32:34.815616 23957 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 12:32:34.815630 23957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:34.815691 23957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:34.815726 23957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:34.815760 23957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:34.815809 23957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:34.820919 23957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:38245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:38245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=40749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:34.821899 23957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:34.822887 23957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:34.831400 23962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:34.831410 23965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:34.831421 23963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:34.831480 23957 server_base.cc:1034] running on GCE node
I20260806 12:32:34.832242 23957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:34.833204 23957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:34.834398 23957 hybrid_clock.cc:648] HybridClock initialized: now 1786019554834394 us; error 38 us; skew 500 ppm
Aug 06 12:32:34 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019554, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:34.837121 23957 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:32:34.838004 23957 webserver.cc:460] Webserver started at http://127.6.152.190:40749/ using document root <none> and password file <none>
I20260806 12:32:34.838529 23957 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:34.838619 23957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:34.842315 23957 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:34.844367 23972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:34.845525 23957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:32:34.845659 23957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "d925806e76ef4db29da179ede05c9a91"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "6eb1b8cdad2e3614313982ee772dacb0"
server_key_iv: "6fbd66cbf2ae26c202a455b21b413c7f"
server_key_version: "encryptionkey@0"
I20260806 12:32:34.846179 23957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:34.881390 23957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:34.884699 23957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:34.884907 23957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:34.892098 23957 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:38245
I20260806 12:32:34.892393 24027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:38245 every 8 connection(s)
I20260806 12:32:34.893178 23957 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:34.897459 24028 sys_catalog.cc:263] Verifying existing consensus state
I20260806 12:32:34.898891 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 23957
I20260806 12:32:34.899104 24028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap starting.
I20260806 12:32:34.899464 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:34393
--local_ip_for_outbound_sockets=127.6.152.129
--tserver_master_addrs=127.6.152.190:38245
--webserver_port=38813
--webserver_interface=127.6.152.129
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:32:34.924268 24028 log.cc:824] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:34.938660 24028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:32:34.939173 24028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Bootstrap complete.
I20260806 12:32:34.944783 24028 raft_consensus.cc:348] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:34.945299 24028 raft_consensus.cc:729] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d925806e76ef4db29da179ede05c9a91, State: Initialized, Role: FOLLOWER
I20260806 12:32:34.945848 24028 consensus_queue.cc:260] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:34.946010 24028 raft_consensus.cc:388] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:34.946087 24028 raft_consensus.cc:482] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:34.946179 24028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 2 FOLLOWER]: Advancing to term 3
I20260806 12:32:34.948611 24028 raft_consensus.cc:504] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:34.948982 24028 leader_election.cc:302] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d925806e76ef4db29da179ede05c9a91; no voters:
I20260806 12:32:34.949337 24028 leader_election.cc:288] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 12:32:34.949461 24033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 12:32:34.949720 24033 raft_consensus.cc:686] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [term 3 LEADER]: Becoming Leader. State: Replica: d925806e76ef4db29da179ede05c9a91, State: Running, Role: LEADER
I20260806 12:32:34.950047 24033 consensus_queue.cc:237] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } }
I20260806 12:32:34.950605 24028 sys_catalog.cc:564] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:34.951362 24034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d925806e76ef4db29da179ede05c9a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } } }
I20260806 12:32:34.951475 24034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:34.952178 24035 sys_catalog.cc:455] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d925806e76ef4db29da179ede05c9a91. Latest consensus state: current_term: 3 leader_uuid: "d925806e76ef4db29da179ede05c9a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d925806e76ef4db29da179ede05c9a91" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 38245 } } }
I20260806 12:32:34.952351 24035 sys_catalog.cc:458] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:34.952633 24040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:34.957638 24040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:34.958345 24040 catalog_manager.cc:1259] Loaded cluster ID: 73b993b7b35645969bcb60968cf53d43
I20260806 12:32:34.958436 24040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:34.960978 24040 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:34.961745 24040 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Loaded TSK: 0
I20260806 12:32:34.961858 24040 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d925806e76ef4db29da179ede05c9a91: Loaded TSK: 1
F20260806 12:32:34.962646 24040 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1fd089c9ad google::LogMessage::Fail() at ??:0
@ 0x7f1fd089f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1fd089c4de google::LogMessage::Flush() at ??:0
@ 0x7f1fd089da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1fd356018d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1fd3558f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1fdb2017bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1fdb1db99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1fdb1dc178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f1fdb21ddf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1fdb23c82b std::function<>::operator()() at ??:0
@ 0x7f1fdb1dbb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1fdb1dcf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1fdb1d9579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1fdb21d640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1fd2aee3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1fd2aeec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1fd2af03f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1fd2adf512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1fd41236db start_thread at ??:0
@ 0x7f1fcf39a61f clone at ??:0
W20260806 12:32:35.066176 24031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:35.066461 24031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:35.066547 24031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:35.071079 24031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:35.071189 24031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:35.071316 24031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:35.077152 24031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:34393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=38813
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:35.078274 24031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:35.079437 24031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:35.090768 24077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:35.090761 24078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:35.091012 24080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:35.091131 24031 server_base.cc:1034] running on GCE node
I20260806 12:32:35.091572 24031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:35.092232 24031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:35.093401 24031 hybrid_clock.cc:648] HybridClock initialized: now 1786019555093369 us; error 46 us; skew 500 ppm
Aug 06 12:32:35 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019555, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:35.096213 24031 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:32:35.097083 24031 webserver.cc:460] Webserver started at http://127.6.152.129:38813/ using document root <none> and password file <none>
I20260806 12:32:35.097622 24031 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:35.097684 24031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:35.101681 24031 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:32:35.103878 24088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.105170 24031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:35.105259 24031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "736ac31fd2bd4acd9161c033535c0d20"
format_stamp: "Formatted at 2026-08-06 12:31:52 on dist-test-slave-1xrh"
server_key: "89a1933c9a14631daf847ff0ede27f31"
server_key_iv: "65f0f2efd502949dd46fa4f6c0849946"
server_key_version: "encryptionkey@0"
I20260806 12:32:35.105793 24031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:35.119971 24031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:35.123027 24031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:35.123255 24031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:35.123894 24031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:35.124950 24096 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 12:32:35.129025 24031 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 12:32:35.129087 24031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 12:32:35.129153 24031 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 12:32:35.130717 24031 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 12:32:35.130774 24031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 12:32:35.131016 24096 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap starting.
I20260806 12:32:35.141821 24031 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:34393
I20260806 12:32:35.142000 24207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:34393 every 8 connection(s)
I20260806 12:32:35.142959 24031 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:35.149509 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24031
I20260806 12:32:35.150025 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:40579
--local_ip_for_outbound_sockets=127.6.152.130
--tserver_master_addrs=127.6.152.190:38245
--webserver_port=40493
--webserver_interface=127.6.152.130
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:32:35.152324 24096 log.cc:824] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:35.161856 24096 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 12:32:35.162343 24096 tablet_bootstrap.cc:492] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Bootstrap complete.
I20260806 12:32:35.164027 24096 ts_tablet_manager.cc:1397] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.011s
I20260806 12:32:35.167229 24096 raft_consensus.cc:348] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:35.167744 24096 raft_consensus.cc:729] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736ac31fd2bd4acd9161c033535c0d20, State: Initialized, Role: FOLLOWER
I20260806 12:32:35.168390 24096 consensus_queue.cc:260] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:35.169425 24096 ts_tablet_manager.cc:1428] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 12:32:35.190897 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:32:35.145376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:35107)
0806 12:32:35.145978 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.146051 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:35.147098 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:35.190114 (+ 43016us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":144,"threads_started":1}
W20260806 12:32:35.191432 24208 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:38245 (0 consecutive failures): Network error: Failed to ping master at 127.6.152.190:38245: Client connection negotiation failed: client connection to 127.6.152.190:38245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 12:32:35.192183 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.191947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:44579)
0806 12:32:35.192069 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.192130 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 12:32:35.192917 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.192689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:33263)
0806 12:32:35.192788 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.192858 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 12:32:35.193143 24208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 12:32:35.268172 24211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:35.268407 24211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:35.268448 24211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:35.272416 24211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:35.272462 24211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:35.272545 24211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:35.277539 24211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:40579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=40493
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:35.278448 24211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:35.279431 24211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:35.288429 24221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:35.288421 24219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:35.288407 24218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:35.288546 24211 server_base.cc:1034] running on GCE node
I20260806 12:32:35.289116 24211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:35.289749 24211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:35.290910 24211 hybrid_clock.cc:648] HybridClock initialized: now 1786019555290896 us; error 36 us; skew 500 ppm
Aug 06 12:32:35 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019555, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:35.293692 24211 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:32:35.294533 24211 webserver.cc:460] Webserver started at http://127.6.152.130:40493/ using document root <none> and password file <none>
I20260806 12:32:35.295053 24211 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:35.295176 24211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:35.298681 24211 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:32:35.300694 24229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.301843 24211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:32:35.301960 24211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "419532933013415da09839e93129881e"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "ec57b580915b93a2682aa3c3d19c3bd5"
server_key_iv: "36d2f5f3dbf62a3e0fff70cd82e9ff5c"
server_key_version: "encryptionkey@0"
I20260806 12:32:35.302461 24211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:35.313375 24211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:35.316816 24211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:35.317006 24211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:35.317552 24211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:35.318333 24211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:35.318408 24211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.318473 24211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:35.318502 24211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.327481 24211 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:40579
I20260806 12:32:35.327522 24354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:40579 every 8 connection(s)
I20260806 12:32:35.328374 24211 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:32:35.331771 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.330332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:35747)
0806 12:32:35.330945 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.331043 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":106,"threads_started":1}
W20260806 12:32:35.332255 24355 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:38245 (0 consecutive failures): Network error: Failed to ping master at 127.6.152.190:38245: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
I20260806 12:32:35.332778 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.332585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:60193)
0806 12:32:35.332685 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.332729 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 12:32:35.333452 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.333214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:33895)
0806 12:32:35.333338 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.333393 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 12:32:35.333626 24355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 12:32:35.337550 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24211
I20260806 12:32:35.338231 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:44807
--local_ip_for_outbound_sockets=127.6.152.131
--tserver_master_addrs=127.6.152.190:38245
--webserver_port=34403
--webserver_interface=127.6.152.131
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:35.448863 24358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:35.449110 24358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:35.449151 24358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:35.453153 24358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:35.453213 24358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:35.453297 24358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:32:35.458592 24358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:44807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=34403
--tserver_master_addrs=127.6.152.190:38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:35.459721 24358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:35.460772 24358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:35.470347 24364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:35.470345 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:35.470527 24358 server_base.cc:1034] running on GCE node
W20260806 12:32:35.470340 24367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:35.471019 24358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:35.471676 24358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:35.472900 24358 hybrid_clock.cc:648] HybridClock initialized: now 1786019555472888 us; error 27 us; skew 500 ppm
Aug 06 12:32:35 dist-test-slave-1xrh krb5kdc[22845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019555, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:35.475703 24358 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:32:35.476423 24358 webserver.cc:460] Webserver started at http://127.6.152.131:34403/ using document root <none> and password file <none>
I20260806 12:32:35.476928 24358 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:35.476984 24358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:35.480495 24358 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:32:35.482316 24375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.483534 24358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:35.483644 24358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "667a68c7828a4fddbcca08c2333b10a5"
format_stamp: "Formatted at 2026-08-06 12:31:53 on dist-test-slave-1xrh"
server_key: "3fdd83035532e1a1aa6ef4194bbed3d9"
server_key_iv: "debd4fb719ee576b225d7e99fb685808"
server_key_version: "encryptionkey@0"
I20260806 12:32:35.484175 24358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:35.508226 24358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:35.511080 24358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:35.511314 24358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:35.511866 24358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:35.512634 24358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:35.512679 24358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.512746 24358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:35.512784 24358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:35.521752 24358 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:44807
I20260806 12:32:35.521826 24491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:44807 every 8 connection(s)
I20260806 12:32:35.522745 24358 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:32:35.525431 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24358
I20260806 12:32:35.526635 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.524826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:59193)
0806 12:32:35.525606 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.525716 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":119,"threads_started":1}
W20260806 12:32:35.527043 24492 heartbeater.cc:644] Failed to heartbeat to 127.6.152.190:38245 (0 consecutive failures): Network error: Failed to ping master at 127.6.152.190:38245: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
I20260806 12:32:35.527590 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.527393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:38735)
0806 12:32:35.527494 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.527538 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 12:32:35.528093 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:35.527953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:56999)
0806 12:32:35.528016 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:35.528051 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 12:32:35.528260 24492 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 12:32:36.194108 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.193696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:46869)
0806 12:32:36.193930 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:36.193995 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260806 12:32:36.295272 24497 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:36.295506 24497 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:36.296614 24497 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 419532933013415da09839e93129881e (127.6.152.130:40579)
Aug 06 12:32:36 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019555, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.130@KRBTEST.COM
Aug 06 12:32:36 dist-test-slave-1xrh krb5kdc[22845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019555, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.131@KRBTEST.COM
I20260806 12:32:36.310494 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.296828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:40579 (local address 127.6.152.129:38313)
0806 12:32:36.297036 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:36.297051 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:36.297161 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:36.298972 (+ 1811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:36.298979 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:36.299552 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:36.300002 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:36.300016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.301304 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.301308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:36.302163 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:36.302173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.302858 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.302862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:36.303038 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:36.303780 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:36.303826 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:36.305380 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:36.307586 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:36.307591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:36.307604 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:36.307853 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:36.308260 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:36.308264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:36.308265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:36.308402 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:36.309402 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:36.309412 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:36.309719 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 12:32:36.309924 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 12:32:36.310338 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":253,"spinlock_wait_cycles":56192}
I20260806 12:32:36.310494 24500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.296828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:44807 (local address 127.6.152.129:49397)
0806 12:32:36.297715 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:36.297729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:36.297864 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:36.299238 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:36.299245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:36.299547 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:36.300002 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:36.300017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.301310 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.301313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:36.302161 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:36.302170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.302856 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.302859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:36.303031 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:36.303873 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:36.303885 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:36.305749 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:36.307923 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:36.307928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:36.307931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:36.308132 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:36.308561 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:36.308564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:36.308566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:36.308690 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:36.309542 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:36.309545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:36.309719 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 12:32:36.309924 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 12:32:36.310266 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":680,"spinlock_wait_cycles":22656,"thread_start_us":66,"threads_started":1}
I20260806 12:32:36.310554 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.297097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:38313 (local address 127.6.152.130:40579)
0806 12:32:36.297411 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:36.297424 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:36.297459 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:32:36.298266 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:36.298274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:36.298624 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:36.298865 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:36.300178 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.301151 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.302290 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.302729 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.302908 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:36.305642 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:36.305647 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:36.305658 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:36.305685 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:36.307467 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:36.307974 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:36.307985 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:36.307989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:36.308142 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:36.308721 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:36.308726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:36.308727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:36.309267 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:36.309434 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:36.310077 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 12:32:36.310367 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":78,"threads_started":1}
I20260806 12:32:36.310849 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.297103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:49397 (local address 127.6.152.131:44807)
0806 12:32:36.297412 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:36.297427 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:36.297845 (+ 418us) server_negotiation.cc:407] Connection header received
0806 12:32:36.298613 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:36.298619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:36.298973 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:36.299292 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:36.300197 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.301150 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.302285 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:36.302730 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:36.302900 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:36.305926 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:36.305957 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:36.305995 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:36.306029 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:36.307802 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:36.308245 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:36.308269 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:36.308274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:36.308414 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:36.308813 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:36.308818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:36.308820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:36.309373 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:36.309497 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:36.310084 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 12:32:36.310367 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":124,"threads_started":1}
W20260806 12:32:36.313179 24093 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
W20260806 12:32:36.313357 24089 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 419532933013415da09839e93129881e (127.6.152.130:40579): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:36.313446 24089 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 736ac31fd2bd4acd9161c033535c0d20; no voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5
I20260806 12:32:36.313740 24497 raft_consensus.cc:2726] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:32:36.334463 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.334102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:58307)
0806 12:32:36.334319 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:36.334371 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
I20260806 12:32:36.529158 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:36.528759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:54937)
0806 12:32:36.529007 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:36.529057 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
I20260806 12:32:37.195331 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:37.194857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:46911)
0806 12:32:37.195165 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:37.195230 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260806 12:32:37.335825 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:37.335338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:47715)
0806 12:32:37.335640 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:37.335699 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":110,"threads_started":1}
I20260806 12:32:37.530269 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:37.529833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:49931)
0806 12:32:37.530112 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:37.530163 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":105,"threads_started":1}
I20260806 12:32:38.196424 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:38.196039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:55223)
0806 12:32:38.196278 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:38.196330 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":71,"threads_started":1}
I20260806 12:32:38.336879 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:38.336482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:35887)
0806 12:32:38.336737 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:38.336788 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260806 12:32:38.452456 24508 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:38.452600 24508 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:38.453032 24508 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 419532933013415da09839e93129881e (127.6.152.130:40579)
W20260806 12:32:38.453881 24089 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 419532933013415da09839e93129881e (127.6.152.130:40579): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
W20260806 12:32:38.454041 24093 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:38.454147 24093 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 736ac31fd2bd4acd9161c033535c0d20; no voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5
I20260806 12:32:38.454346 24508 raft_consensus.cc:2726] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:32:38.531520 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:38.531090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:37413)
0806 12:32:38.531369 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:38.531420 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":124,"threads_started":1}
I20260806 12:32:39.197613 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:39.197214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:52011)
0806 12:32:39.197461 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:39.197515 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260806 12:32:39.338059 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:39.337646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:42371)
0806 12:32:39.337922 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:39.337976 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":93,"threads_started":1}
I20260806 12:32:39.532724 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:39.532294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:42331)
0806 12:32:39.532585 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:39.532637 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":123,"threads_started":1}
I20260806 12:32:40.199009 24514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:40.198489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:40325)
0806 12:32:40.198789 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:40.198848 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":72,"threads_started":1}
I20260806 12:32:40.339380 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:40.338919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:49399)
0806 12:32:40.339234 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:40.339286 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":87,"threads_started":1}
I20260806 12:32:40.533838 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:40.533452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:37587)
0806 12:32:40.533698 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:40.533748 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
I20260806 12:32:41.200227 24518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:41.199788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:39909)
0806 12:32:41.200068 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:41.200119 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":121,"threads_started":1}
I20260806 12:32:41.340559 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:41.340172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:58009)
0806 12:32:41.340409 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:41.340463 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260806 12:32:41.443007 24520 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:41.443259 24520 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:41.443768 24520 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 419532933013415da09839e93129881e (127.6.152.130:40579)
W20260806 12:32:41.444661 24093 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
W20260806 12:32:41.444778 24089 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 419532933013415da09839e93129881e (127.6.152.130:40579): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:41.444913 24089 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 736ac31fd2bd4acd9161c033535c0d20; no voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5
I20260806 12:32:41.445060 24520 raft_consensus.cc:2726] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:32:41.534961 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:41.534553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:53741)
0806 12:32:41.534818 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:41.534873 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260806 12:32:42.201406 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:42.201004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:60621)
0806 12:32:42.201246 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:42.201300 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":82,"threads_started":1}
I20260806 12:32:42.341751 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:42.341308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:54729)
0806 12:32:42.341603 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:42.341658 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":72,"threads_started":1}
I20260806 12:32:42.536252 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:42.535714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:45783)
0806 12:32:42.536095 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:42.536163 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":85,"threads_started":1}
I20260806 12:32:43.202570 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:43.202134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:52165)
0806 12:32:43.202413 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:43.202468 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
I20260806 12:32:43.342865 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:43.342454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:48805)
0806 12:32:43.342711 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:43.342765 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260806 12:32:43.537366 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:43.536979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:41817)
0806 12:32:43.537221 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:43.537274 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260806 12:32:44.203701 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:44.203311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:56119)
0806 12:32:44.203548 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:44.203602 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260806 12:32:44.343947 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:44.343526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:58941)
0806 12:32:44.343769 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:44.343849 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260806 12:32:44.538534 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:44.538097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:44031)
0806 12:32:44.538372 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:44.538441 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
I20260806 12:32:45.204983 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:45.204571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:41323)
0806 12:32:45.204818 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:45.204871 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260806 12:32:45.345104 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:45.344712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:53531)
0806 12:32:45.344950 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:45.345004 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260806 12:32:45.496424 24534 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:45.496562 24534 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:45.496949 24534 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 419532933013415da09839e93129881e (127.6.152.130:40579)
W20260806 12:32:45.497699 24093 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
W20260806 12:32:45.497789 24089 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 419532933013415da09839e93129881e (127.6.152.130:40579): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:45.497901 24089 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 736ac31fd2bd4acd9161c033535c0d20; no voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5
I20260806 12:32:45.498086 24534 raft_consensus.cc:2726] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:32:45.539839 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:45.539368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:57481)
0806 12:32:45.539698 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:45.539750 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260806 12:32:46.206233 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:46.205835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:54965)
0806 12:32:46.206081 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:46.206136 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260806 12:32:46.346246 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:46.345824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:60015)
0806 12:32:46.346099 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:46.346152 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260806 12:32:46.541119 24538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:46.540522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:60087)
0806 12:32:46.540927 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:46.540993 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":214,"threads_started":1}
I20260806 12:32:47.207435 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:47.206940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:55307)
0806 12:32:47.207268 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:47.207326 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":73,"threads_started":1}
I20260806 12:32:47.347489 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:47.346997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:48639)
0806 12:32:47.347332 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:47.347390 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":139,"threads_started":1}
I20260806 12:32:47.542222 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:47.541807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:59877)
0806 12:32:47.542057 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:47.542108 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
I20260806 12:32:48.019855 24542 raft_consensus.cc:482] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:48.019991 24542 raft_consensus.cc:504] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "667a68c7828a4fddbcca08c2333b10a5" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 44807 } } peers { permanent_uuid: "736ac31fd2bd4acd9161c033535c0d20" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 34393 } } peers { permanent_uuid: "419532933013415da09839e93129881e" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 40579 } }
I20260806 12:32:48.020540 24542 leader_election.cc:288] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807), 419532933013415da09839e93129881e (127.6.152.130:40579)
W20260806 12:32:48.021271 24093 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 667a68c7828a4fddbcca08c2333b10a5 (127.6.152.131:44807): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
W20260806 12:32:48.021360 24089 leader_election.cc:341] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 419532933013415da09839e93129881e (127.6.152.130:40579): Not found: Tablet not found: 92a1fa3f8d5d43e293c9112227976e62
I20260806 12:32:48.021489 24089 leader_election.cc:302] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 736ac31fd2bd4acd9161c033535c0d20; no voters: 419532933013415da09839e93129881e, 667a68c7828a4fddbcca08c2333b10a5
I20260806 12:32:48.021677 24542 raft_consensus.cc:2726] T 92a1fa3f8d5d43e293c9112227976e62 P 736ac31fd2bd4acd9161c033535c0d20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 12:32:48.208583 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:48.208165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:38755)
0806 12:32:48.208429 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:48.208485 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":72,"threads_started":1}
I20260806 12:32:48.348618 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:48.348225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:49743)
0806 12:32:48.348472 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:48.348525 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":91,"threads_started":1}
I20260806 12:32:48.543349 24545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:48.542960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:34469)
0806 12:32:48.543210 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:48.543261 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260806 12:32:49.209826 24546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:49.209430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:52273)
0806 12:32:49.209674 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:49.209729 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260806 12:32:49.349788 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:49.349373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:54611)
0806 12:32:49.349645 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:49.349696 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260806 12:32:49.544545 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:49.544124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.131:37063)
0806 12:32:49.544387 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:49.544452 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260806 12:32:50.211213 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:50.210761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.129:46631)
0806 12:32:50.211018 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:50.211072 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
I20260806 12:32:50.351016 24550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:50.350603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:38245 (local address 127.6.152.130:48539)
0806 12:32:50.350851 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:50.350902 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.190:38245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260806 12:32:50.531556 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24031
I20260806 12:32:50.539278 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24211
I20260806 12:32:50.545288 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24358
2026-08-06T12:32:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59356 ms)
[----------] 30 tests from SecurityITest (246166 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:32:50 dist-test-slave-1xrh krb5kdc[24553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:32:50 dist-test-slave-1xrh krb5kdc[24553](info): set up 2 sockets
Aug 06 12:32:50 dist-test-slave-1xrh krb5kdc[24553](info): commencing operation
krb5kdc: starting...
W20260806 12:32:51.704244 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:32:51 dist-test-slave-1xrh krb5kdc[24553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:32:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:32:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:32:51.826812 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35223
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:51.940611 24566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:51.940981 24566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:51.941058 24566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:51.945710 24566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:51.945765 24566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:51.945783 24566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:51.945829 24566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:51.945860 24566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:51.951555 24566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:51.952558 24566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:51.953537 24566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:51.962374 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:51.962532 24566 server_base.cc:1034] running on GCE node
W20260806 12:32:51.962371 24571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:51.962759 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:51.963402 24566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:51.964491 24566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:51.965654 24566 hybrid_clock.cc:648] HybridClock initialized: now 1786019571965643 us; error 25 us; skew 500 ppm
Aug 06 12:32:51 dist-test-slave-1xrh krb5kdc[24553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:51.968398 24566 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:32:51.969226 24566 webserver.cc:460] Webserver started at http://127.6.152.190:33757/ using document root <none> and password file <none>
I20260806 12:32:51.969718 24566 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:51.969774 24566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:51.969939 24566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:51.971917 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "718598f9623b469cbf6d83e1d7eb1373"
format_stamp: "Formatted at 2026-08-06 12:32:51 on dist-test-slave-1xrh"
server_key: "ea5051690b02c09d26ad13ca7dd3feb3"
server_key_iv: "ff6d640a04c9830a59a8f6b6fd51867d"
server_key_version: "encryptionkey@0"
I20260806 12:32:51.972416 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "718598f9623b469cbf6d83e1d7eb1373"
format_stamp: "Formatted at 2026-08-06 12:32:51 on dist-test-slave-1xrh"
server_key: "ea5051690b02c09d26ad13ca7dd3feb3"
server_key_iv: "ff6d640a04c9830a59a8f6b6fd51867d"
server_key_version: "encryptionkey@0"
I20260806 12:32:51.976063 24566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:32:51.978307 24581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:51.979542 24566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:32:51.979673 24566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "718598f9623b469cbf6d83e1d7eb1373"
format_stamp: "Formatted at 2026-08-06 12:32:51 on dist-test-slave-1xrh"
server_key: "ea5051690b02c09d26ad13ca7dd3feb3"
server_key_iv: "ff6d640a04c9830a59a8f6b6fd51867d"
server_key_version: "encryptionkey@0"
I20260806 12:32:51.979780 24566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:52.005851 24566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:52.008872 24566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:52.009076 24566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:52.016219 24566 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:35223
I20260806 12:32:52.016247 24638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:35223 every 8 connection(s)
I20260806 12:32:52.017134 24566 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:52.019799 24639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:52.023422 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24566
I20260806 12:32:52.023597 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:32:52.023874 6754 external_mini_cluster.cc:1428] Setting key c07a7b432128eab70c8739e057f9d499
I20260806 12:32:52.025578 24639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373: Bootstrap starting.
I20260806 12:32:52.027776 24639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:52.028658 24639 log.cc:824] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:52.030573 24639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373: No bootstrap required, opened a new log
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:52.034770 24639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718598f9623b469cbf6d83e1d7eb1373" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35223 } }
I20260806 12:32:52.035023 24639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:52.035100 24639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718598f9623b469cbf6d83e1d7eb1373, State: Initialized, Role: FOLLOWER
I20260806 12:32:52.035619 24639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718598f9623b469cbf6d83e1d7eb1373" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35223 } }
I20260806 12:32:52.035761 24639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:52.035849 24639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:52.035951 24639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:52.037055 24639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718598f9623b469cbf6d83e1d7eb1373" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35223 } }
I20260806 12:32:52.037422 24639 leader_election.cc:302] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 718598f9623b469cbf6d83e1d7eb1373; no voters:
I20260806 12:32:52.037783 24639 leader_election.cc:288] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:32:52.037895 24645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:52.038178 24645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [term 1 LEADER]: Becoming Leader. State: Replica: 718598f9623b469cbf6d83e1d7eb1373, State: Running, Role: LEADER
I20260806 12:32:52.038539 24645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718598f9623b469cbf6d83e1d7eb1373" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35223 } }
I20260806 12:32:52.038651 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.025281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.6.152.190:35223)
0806 12:32:52.025751 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.025775 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.025809 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:32:52.026413 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.026428 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.026735 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.027094 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:52.027974 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.029361 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.030222 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.030905 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.031065 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.033481 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:52.033487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:52.033503 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:52.033534 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:52.035494 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.035994 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.036003 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.036009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.036157 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.036430 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.036433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.036435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.036952 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:52.037156 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.037414 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.037715 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":135,"threads_started":1}
I20260806 12:32:52.039016 24639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:52.040190 24646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "718598f9623b469cbf6d83e1d7eb1373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718598f9623b469cbf6d83e1d7eb1373" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35223 } } }
I20260806 12:32:52.040307 24646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:52.041605 24654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:52.041553 24647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718598f9623b469cbf6d83e1d7eb1373. Latest consensus state: current_term: 1 leader_uuid: "718598f9623b469cbf6d83e1d7eb1373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718598f9623b469cbf6d83e1d7eb1373" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35223 } } }
I20260806 12:32:52.042048 24647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:52.044479 24654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:52.049283 24654 catalog_manager.cc:1347] Generated new cluster ID: 8419fef619934fa2b9ee33f587ec64c2
I20260806 12:32:52.049345 24654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:52.062516 24654 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:32:52.063437 24654 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:52.074488 24654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373: Generated new TSK 0
I20260806 12:32:52.075209 24654 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:32:52.105792 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35223
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:52.217684 24667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:52.217974 24667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:52.218072 24667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:52.222064 24667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:52.222159 24667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:52.222249 24667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:52.227943 24667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:52.228965 24667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:52.229986 24667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:52.239099 24675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:52.239178 24673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:52.239169 24672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:52.239293 24667 server_base.cc:1034] running on GCE node
I20260806 12:32:52.239737 24667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:52.240368 24667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:52.241549 24667 hybrid_clock.cc:648] HybridClock initialized: now 1786019572241519 us; error 48 us; skew 500 ppm
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:52.244443 24667 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:32:52.245244 24667 webserver.cc:460] Webserver started at http://127.6.152.129:38213/ using document root <none> and password file <none>
I20260806 12:32:52.245752 24667 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:52.245838 24667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:52.246085 24667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:52.248076 24667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "23999d1943cb4908af42dec716764c28"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "c42f94cf4ec2e39fdcd98a6ded3b0318"
server_key_iv: "60c297bddec15438f14c17275d6f91c8"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.248644 24667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "23999d1943cb4908af42dec716764c28"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "c42f94cf4ec2e39fdcd98a6ded3b0318"
server_key_iv: "60c297bddec15438f14c17275d6f91c8"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.252305 24667 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:32:52.254566 24682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.255693 24667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:32:52.255823 24667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "23999d1943cb4908af42dec716764c28"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "c42f94cf4ec2e39fdcd98a6ded3b0318"
server_key_iv: "60c297bddec15438f14c17275d6f91c8"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.255930 24667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:52.281158 24667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:52.284647 24667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:52.284821 24667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:52.285363 24667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:52.286116 24667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:52.286163 24667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.286201 24667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:52.286221 24667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.294900 24667 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:37529
I20260806 12:32:52.294970 24801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:37529 every 8 connection(s)
I20260806 12:32:52.295773 24667 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:52.302314 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24667
I20260806 12:32:52.302465 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:32:52.302767 6754 external_mini_cluster.cc:1428] Setting key ee05bee564e8c9b5f6f3a047c7112932
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:52.309080 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.297452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:43271 (local address 127.6.152.190:35223)
0806 12:32:52.297584 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.297587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.298324 (+ 737us) server_negotiation.cc:407] Connection header received
0806 12:32:52.299237 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.299240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.299313 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.299414 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:52.300503 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.301032 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.302149 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.302383 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.302494 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.305212 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:52.305215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:52.305217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:52.305239 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:52.306864 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.307416 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.307421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.307423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.307528 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.307867 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.307873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.307878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.308134 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:52.308237 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.308694 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.308837 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:32:52.309813 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.297751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35223 (local address 127.6.152.129:43271)
0806 12:32:52.298180 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:52.298219 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:52.299005 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:52.299575 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:52.299581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:52.299895 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:52.300352 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.300364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.301159 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.301168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.301996 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.302005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.302530 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.302535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.302690 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.303319 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:52.303342 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:52.305037 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:52.306990 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:52.306994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:52.307004 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:52.307318 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:52.307628 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:52.307630 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:52.307632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:52.307756 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:52.308235 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:52.308239 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:52.308456 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 12:32:52.308594 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 12:32:52.308858 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260806 12:32:52.310990 24802 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35223
I20260806 12:32:52.311280 24802 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:52.311842 24802 heartbeater.cc:511] Master 127.6.152.190:35223 requested a full tablet report, sending...
I20260806 12:32:52.313359 24600 ts_manager.cc:194] Registered new tserver with Master: 23999d1943cb4908af42dec716764c28 (127.6.152.129:37529)
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:32:52.314546 24600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:43271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:32:52.332176 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35223
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:52.440428 24807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:52.440693 24807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:52.440793 24807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:52.444766 24807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:52.444844 24807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:52.444931 24807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:52.450083 24807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:52.451066 24807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:52.452140 24807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:52.461854 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:52.461845 24807 server_base.cc:1034] running on GCE node
W20260806 12:32:52.461826 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:52.461876 24821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:52.462491 24807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:52.463074 24807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:52.464277 24807 hybrid_clock.cc:648] HybridClock initialized: now 1786019572464275 us; error 41 us; skew 500 ppm
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:52.466997 24807 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:32:52.467885 24807 webserver.cc:460] Webserver started at http://127.6.152.130:37027/ using document root <none> and password file <none>
I20260806 12:32:52.468427 24807 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:52.468523 24807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:52.468767 24807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:52.470682 24807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "bfd0548e8221422d821351e6350e01b2"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "27327b1f9a26eb01020b777fc9e8dba2"
server_key_iv: "1f1e7a717dd93ac71c65068f217528e4"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.471297 24807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "bfd0548e8221422d821351e6350e01b2"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "27327b1f9a26eb01020b777fc9e8dba2"
server_key_iv: "1f1e7a717dd93ac71c65068f217528e4"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.474817 24807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:32:52.477331 24828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.478572 24807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 12:32:52.478681 24807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "bfd0548e8221422d821351e6350e01b2"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "27327b1f9a26eb01020b777fc9e8dba2"
server_key_iv: "1f1e7a717dd93ac71c65068f217528e4"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.478766 24807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:52.490633 24807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:52.493177 24807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:52.493311 24807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:52.493774 24807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:52.494475 24807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:52.494520 24807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.494555 24807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:52.494578 24807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.503175 24807 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:44973
I20260806 12:32:52.503196 24947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:44973 every 8 connection(s)
I20260806 12:32:52.504070 24807 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:32:52.507884 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24807
I20260806 12:32:52.508069 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:32:52.508432 6754 external_mini_cluster.cc:1428] Setting key 0d185135b00cc12b28215d55e3c2f188
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
I20260806 12:32:52.519156 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.505873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:39463 (local address 127.6.152.190:35223)
0806 12:32:52.506038 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.506043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.506865 (+ 822us) server_negotiation.cc:407] Connection header received
0806 12:32:52.507860 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.507863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.507910 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.507987 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:52.509530 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.510103 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.511894 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.512346 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.512494 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.515240 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:52.515244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:52.515246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:52.515270 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:52.516878 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.517463 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.517467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.517469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.517558 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.517947 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.517950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.517952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.518150 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:52.518273 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.518796 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.518930 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 12:32:52.519958 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.506276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35223 (local address 127.6.152.130:39463)
0806 12:32:52.506714 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:52.506756 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:52.507581 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:52.508224 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:52.508234 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:52.508730 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:52.509373 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.509405 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.510230 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.510234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.511730 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.511745 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.512509 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.512512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.512650 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.513255 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:52.513270 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:52.515006 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:52.517025 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:52.517031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:52.517045 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:52.517342 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:52.517663 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:52.517667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:52.517670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:52.517829 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:52.518272 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:52.518279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:52.518527 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 12:32:52.518728 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 12:32:52.519081 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260806 12:32:52.521399 24948 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35223
I20260806 12:32:52.521652 24948 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:52.522126 24948 heartbeater.cc:511] Master 127.6.152.190:35223 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
I20260806 12:32:52.523368 24600 ts_manager.cc:194] Registered new tserver with Master: bfd0548e8221422d821351e6350e01b2 (127.6.152.130:44973)
I20260806 12:32:52.524240 24600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:39463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:32:52.538164 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35223
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:52.649420 24955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:52.649673 24955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:52.649713 24955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:52.653764 24955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:52.653882 24955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:52.653995 24955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:32:52.659180 24955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:52.660190 24955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:52.661168 24955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:52.670693 24961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:52.670641 24963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:52.670737 24955 server_base.cc:1034] running on GCE node
W20260806 12:32:52.670641 24960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:52.671278 24955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:52.671895 24955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:52.673090 24955 hybrid_clock.cc:648] HybridClock initialized: now 1786019572673083 us; error 29 us; skew 500 ppm
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:52.681665 24955 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:32:52.682628 24955 webserver.cc:460] Webserver started at http://127.6.152.131:36645/ using document root <none> and password file <none>
I20260806 12:32:52.683341 24955 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:52.683434 24955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:52.683677 24955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:52.685863 24955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "431c07b540a94ef8b91237b3ba12a2fe"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "7e0a0e1b2bb4fffaa4e7cc1b90ed5437"
server_key_iv: "2d6996f260a351414bf8cc1cd07d6539"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.686465 24955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "431c07b540a94ef8b91237b3ba12a2fe"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "7e0a0e1b2bb4fffaa4e7cc1b90ed5437"
server_key_iv: "2d6996f260a351414bf8cc1cd07d6539"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.690176 24955 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:32:52.692585 24970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.693749 24955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:32:52.693888 24955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "431c07b540a94ef8b91237b3ba12a2fe"
format_stamp: "Formatted at 2026-08-06 12:32:52 on dist-test-slave-1xrh"
server_key: "7e0a0e1b2bb4fffaa4e7cc1b90ed5437"
server_key_iv: "2d6996f260a351414bf8cc1cd07d6539"
server_key_version: "encryptionkey@0"
I20260806 12:32:52.694020 24955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:52.726197 24955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:52.729545 24955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:52.729750 24955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:52.730392 24955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:52.731377 24955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:52.731439 24955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.731494 24955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:52.731525 24955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:52.742921 24955 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:35785
I20260806 12:32:52.742944 25088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:35785 every 8 connection(s)
I20260806 12:32:52.743891 24955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:32:52.744689 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 24955
I20260806 12:32:52.744843 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:32:52.745150 6754 external_mini_cluster.cc:1428] Setting key 54202431019ed5d08ecde631bac77e1d
Aug 06 12:32:52 dist-test-slave-1xrh krb5kdc[24553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019572, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:52.757575 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.745856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:49135 (local address 127.6.152.190:35223)
0806 12:32:52.746078 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.746082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.747735 (+ 1653us) server_negotiation.cc:407] Connection header received
0806 12:32:52.748516 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.748518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.748563 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.748639 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:52.749708 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.750275 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.751495 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.751709 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.751795 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.754169 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:52.754172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:52.754173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:52.754196 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:52.755746 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.756207 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.756210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.756211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.756284 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.756610 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.756613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.756614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.756785 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:52.756881 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.757296 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.757406 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260806 12:32:52.758427 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.746325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35223 (local address 127.6.152.131:49135)
0806 12:32:52.747582 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:52.747624 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:52.748346 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:52.748778 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:52.748784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:52.749123 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:52.749575 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.749588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.750385 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.750388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.751340 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.751350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.751838 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.751840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.751977 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.752569 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:52.752585 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:52.754015 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:52.755859 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:52.755864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:52.755874 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:52.756120 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:52.756394 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:52.756396 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:52.756397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:52.756524 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:52.756878 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:52.756882 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:52.757094 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 12:32:52.757236 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 12:32:52.757463 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"thread_start_us":151,"threads_started":1}
I20260806 12:32:52.759486 25089 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35223
I20260806 12:32:52.759725 25089 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:52.760193 25089 heartbeater.cc:511] Master 127.6.152.190:35223 requested a full tablet report, sending...
I20260806 12:32:52.761214 24600 ts_manager.cc:194] Registered new tserver with Master: 431c07b540a94ef8b91237b3ba12a2fe (127.6.152.131:35785)
I20260806 12:32:52.761816 24600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:49135
I20260806 12:32:52.770972 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:32:52.780052 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.772979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.6.152.190:35223)
0806 12:32:52.773132 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.773137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.773278 (+ 141us) server_negotiation.cc:407] Connection header received
0806 12:32:52.773372 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.773375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.773417 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.773488 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:52.774138 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.774693 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.775713 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.775924 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.776023 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.777276 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:52.777279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:52.777281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:52.777302 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:52.778502 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.778868 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.778870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.778871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.778956 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:52.779228 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:52.779230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:52.779231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:52.779431 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:52.779538 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.779732 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.779819 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:32:52.782857 24600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:32:52.786093 24600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:32:52.800257 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.794123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35785 (local address 127.0.0.1:55344)
0806 12:32:52.794465 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:52.794509 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:52.794628 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:52.795785 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:52.795793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:52.795815 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:52.795970 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.795976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.797304 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.797308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.798502 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.798513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.799699 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.799703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.799809 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.799858 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 12:32:52.799991 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 12:32:52.800105 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":25,"thread_start_us":99,"threads_started":1}
I20260806 12:32:52.800274 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.794832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.6.152.131:35785)
0806 12:32:52.795237 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.795251 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.795277 (+ 26us) server_negotiation.cc:407] Connection header received
0806 12:32:52.795370 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.795374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.795599 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.796107 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:52.796268 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.797167 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.798647 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.799572 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.799682 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.799920 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.800032 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.800134 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":131,"threads_started":1}
I20260806 12:32:52.800834 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.794194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:44973 (local address 127.0.0.1:50014)
0806 12:32:52.795107 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:52.795121 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:52.795227 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:52.796278 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:52.796281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:52.796301 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:52.796392 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.796399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.797829 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.797833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.798970 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.798980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.800454 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.800460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.800549 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.800570 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 12:32:52.800638 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 12:32:52.800707 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":4,"thread_start_us":103,"threads_started":1}
I20260806 12:32:52.801121 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.794483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37529 (local address 127.0.0.1:52636)
0806 12:32:52.795538 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:52.795553 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:52.795635 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:52.796505 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:52.796507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:52.796525 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:52.796610 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.796618 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.798189 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.798195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.799275 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:52.799286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.800724 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.800727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:52.800818 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.800838 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 12:32:52.800916 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:32:52.801009 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":92,"threads_started":1}
I20260806 12:32:52.801297 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.795346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50014 (local address 127.6.152.130:44973)
0806 12:32:52.795815 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.795827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.795848 (+ 21us) server_negotiation.cc:407] Connection header received
0806 12:32:52.795947 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.795953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.796131 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.796287 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:52.796521 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.797686 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.799095 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.799916 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.800017 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.800138 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.801055 (+ 917us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.801144 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":154,"threads_started":1}
I20260806 12:32:52.801548 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:52.794926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.6.152.129:37529)
0806 12:32:52.795957 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:52.795972 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:52.795992 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:32:52.796059 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:52.796063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:52.796218 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:52.796346 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:52.796774 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.797849 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.799416 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:52.800271 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:52.800538 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:52.800676 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:52.801098 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 12:32:52.801404 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":130,"threads_started":1}
I20260806 12:32:52.803061 25021 tablet_service.cc:1467] Processing CreateTablet for tablet 562aaf1fdbc94305b650bd7788675b06 (DEFAULT_TABLE table=test-table [id=9b8772841e7347619da99e8f97d842d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:52.803494 24879 tablet_service.cc:1467] Processing CreateTablet for tablet 562aaf1fdbc94305b650bd7788675b06 (DEFAULT_TABLE table=test-table [id=9b8772841e7347619da99e8f97d842d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:52.804157 25021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 562aaf1fdbc94305b650bd7788675b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:52.804375 24879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 562aaf1fdbc94305b650bd7788675b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:52.810040 25106 tablet_bootstrap.cc:492] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Bootstrap starting.
I20260806 12:32:52.810503 25107 tablet_bootstrap.cc:492] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Bootstrap starting.
I20260806 12:32:52.812412 25107 tablet_bootstrap.cc:654] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:52.812419 24734 tablet_service.cc:1467] Processing CreateTablet for tablet 562aaf1fdbc94305b650bd7788675b06 (DEFAULT_TABLE table=test-table [id=9b8772841e7347619da99e8f97d842d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:52.812990 25106 tablet_bootstrap.cc:654] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:52.813503 24734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 562aaf1fdbc94305b650bd7788675b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:52.813576 25107 log.cc:824] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:52.813791 25106 log.cc:824] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:52.815501 25107 tablet_bootstrap.cc:492] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: No bootstrap required, opened a new log
I20260806 12:32:52.815706 25107 ts_tablet_manager.cc:1397] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:32:52.815830 25106 tablet_bootstrap.cc:492] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: No bootstrap required, opened a new log
I20260806 12:32:52.815994 25106 ts_tablet_manager.cc:1397] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:32:52.818575 25107 raft_consensus.cc:348] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:52.818768 25107 raft_consensus.cc:374] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:52.818799 25107 raft_consensus.cc:729] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431c07b540a94ef8b91237b3ba12a2fe, State: Initialized, Role: FOLLOWER
I20260806 12:32:52.819232 25107 consensus_queue.cc:260] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:52.819839 25089 heartbeater.cc:503] Master 127.6.152.190:35223 was elected leader, sending a full tablet report...
I20260806 12:32:52.819743 25106 raft_consensus.cc:348] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:52.819978 25106 raft_consensus.cc:374] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:52.820032 25106 raft_consensus.cc:729] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd0548e8221422d821351e6350e01b2, State: Initialized, Role: FOLLOWER
I20260806 12:32:52.820179 25107 ts_tablet_manager.cc:1428] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 12:32:52.820436 25106 consensus_queue.cc:260] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:52.820747 25111 tablet_bootstrap.cc:492] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: Bootstrap starting.
I20260806 12:32:52.821369 25106 ts_tablet_manager.cc:1428] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:32:52.821419 24948 heartbeater.cc:503] Master 127.6.152.190:35223 was elected leader, sending a full tablet report...
I20260806 12:32:52.822947 25111 tablet_bootstrap.cc:654] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:52.823841 25111 log.cc:824] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:52.825676 25111 tablet_bootstrap.cc:492] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: No bootstrap required, opened a new log
I20260806 12:32:52.825855 25111 ts_tablet_manager.cc:1397] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:32:52.828950 25111 raft_consensus.cc:348] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:52.829133 25111 raft_consensus.cc:374] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:52.829213 25111 raft_consensus.cc:729] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23999d1943cb4908af42dec716764c28, State: Initialized, Role: FOLLOWER
I20260806 12:32:52.829625 25111 consensus_queue.cc:260] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:52.830216 24802 heartbeater.cc:503] Master 127.6.152.190:35223 was elected leader, sending a full tablet report...
I20260806 12:32:52.830477 25111 ts_tablet_manager.cc:1428] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
W20260806 12:32:52.995029 25090 tablet.cc:2369] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:32:53.005227 24949 tablet.cc:2369] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:53.018682 25112 raft_consensus.cc:482] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:53.018891 25112 raft_consensus.cc:504] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:53.020077 25112 leader_election.cc:288] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431c07b540a94ef8b91237b3ba12a2fe (127.6.152.131:35785), 23999d1943cb4908af42dec716764c28 (127.6.152.129:37529)
I20260806 12:32:53.025379 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.020823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:59593 (local address 127.6.152.129:37529)
0806 12:32:53.020964 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.020968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.020993 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:32:53.021059 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.021064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.021119 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.021193 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:53.021560 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.022032 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.024018 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.024698 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.024818 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.024872 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.025163 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.025250 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:32:53.025784 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.020351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37529 (local address 127.6.152.130:59593)
0806 12:32:53.020697 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.020713 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.020805 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.021293 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.021296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.021315 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:53.021433 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.021438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.022141 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.022147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.023867 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.023879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.024824 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.024828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.024966 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.024998 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.025542 (+ 544us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.025624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":155,"thread_start_us":95,"threads_started":1}
I20260806 12:32:53.026217 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.020299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:35785 (local address 127.6.152.130:59077)
0806 12:32:53.020501 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.020519 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.020624 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.021109 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.021112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.021131 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:53.021269 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.021278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.022283 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.022285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.023867 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.023879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.025810 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.025814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.025934 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.025956 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.026034 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.026103 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":167}
I20260806 12:32:53.026233 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.020466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:59077 (local address 127.6.152.131:35785)
0806 12:32:53.020815 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.020821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.020841 (+ 20us) server_negotiation.cc:407] Connection header received
0806 12:32:53.020921 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.020925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.020990 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.021069 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:53.021381 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.022155 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.024947 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.025593 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.025704 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.025749 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.026069 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.026147 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250}
I20260806 12:32:53.026881 24754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562aaf1fdbc94305b650bd7788675b06" candidate_uuid: "bfd0548e8221422d821351e6350e01b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23999d1943cb4908af42dec716764c28" is_pre_election: true
I20260806 12:32:53.027060 25041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562aaf1fdbc94305b650bd7788675b06" candidate_uuid: "bfd0548e8221422d821351e6350e01b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431c07b540a94ef8b91237b3ba12a2fe" is_pre_election: true
I20260806 12:32:53.027221 24754 raft_consensus.cc:2454] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd0548e8221422d821351e6350e01b2 in term 0.
I20260806 12:32:53.027354 25041 raft_consensus.cc:2454] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd0548e8221422d821351e6350e01b2 in term 0.
I20260806 12:32:53.027705 24829 leader_election.cc:302] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23999d1943cb4908af42dec716764c28, bfd0548e8221422d821351e6350e01b2; no voters:
I20260806 12:32:53.027993 25112 raft_consensus.cc:2781] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:32:53.028076 25112 raft_consensus.cc:482] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:32:53.028147 25112 raft_consensus.cc:3037] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:53.029165 25112 raft_consensus.cc:504] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:53.029469 25112 leader_election.cc:288] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [CANDIDATE]: Term 1 election: Requested vote from peers 431c07b540a94ef8b91237b3ba12a2fe (127.6.152.131:35785), 23999d1943cb4908af42dec716764c28 (127.6.152.129:37529)
I20260806 12:32:53.029757 25041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562aaf1fdbc94305b650bd7788675b06" candidate_uuid: "bfd0548e8221422d821351e6350e01b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431c07b540a94ef8b91237b3ba12a2fe"
I20260806 12:32:53.029815 24754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562aaf1fdbc94305b650bd7788675b06" candidate_uuid: "bfd0548e8221422d821351e6350e01b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23999d1943cb4908af42dec716764c28"
I20260806 12:32:53.029919 25041 raft_consensus.cc:3037] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:53.029964 24754 raft_consensus.cc:3037] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:53.030946 25041 raft_consensus.cc:2454] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd0548e8221422d821351e6350e01b2 in term 1.
I20260806 12:32:53.031030 24754 raft_consensus.cc:2454] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd0548e8221422d821351e6350e01b2 in term 1.
I20260806 12:32:53.031283 24831 leader_election.cc:302] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431c07b540a94ef8b91237b3ba12a2fe, bfd0548e8221422d821351e6350e01b2; no voters:
I20260806 12:32:53.031483 25112 raft_consensus.cc:2781] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:53.031735 25112 raft_consensus.cc:686] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 1 LEADER]: Becoming Leader. State: Replica: bfd0548e8221422d821351e6350e01b2, State: Running, Role: LEADER
I20260806 12:32:53.032070 25112 consensus_queue.cc:237] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } }
I20260806 12:32:53.034982 24598 catalog_manager.cc:5495] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfd0548e8221422d821351e6350e01b2 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "bfd0548e8221422d821351e6350e01b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfd0548e8221422d821351e6350e01b2" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 44973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 } health_report { overall_health: UNKNOWN } } }
W20260806 12:32:53.047015 24803 tablet.cc:2369] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:53.090350 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.086668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.6.152.130:44973)
0806 12:32:53.086819 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.086824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.086839 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:32:53.086984 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.086986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.087044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.087181 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:32:53.087445 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.087964 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.088845 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.089074 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.089188 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.089525 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:32:53.089935 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:32:53.090072 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.090116 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.090172 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:32:53.096009 24754 raft_consensus.cc:1261] T 562aaf1fdbc94305b650bd7788675b06 P 23999d1943cb4908af42dec716764c28 [term 1 FOLLOWER]: Refusing update from remote peer bfd0548e8221422d821351e6350e01b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:53.096009 25041 raft_consensus.cc:1261] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 1 FOLLOWER]: Refusing update from remote peer bfd0548e8221422d821351e6350e01b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:53.096751 25112 consensus_queue.cc:1035] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "431c07b540a94ef8b91237b3ba12a2fe" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:53.096987 25116 consensus_queue.cc:1035] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23999d1943cb4908af42dec716764c28" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:53.103207 25121 mvcc.cc:205] Tried to move back new op lower bound from 7315536171394240512 to 7315536171141902336. Current Snapshot: MvccSnapshot[applied={T|T < 7315536171394240512}]
I20260806 12:32:53.105963 25119 mvcc.cc:205] Tried to move back new op lower bound from 7315536171394240512 to 7315536171141902336. Current Snapshot: MvccSnapshot[applied={T|T < 7315536171394240512}]
I20260806 12:32:53.113704 24598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:53.113838 24598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:53.116163 24598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 718598f9623b469cbf6d83e1d7eb1373: Sending DeleteTablet for 3 replicas of tablet 562aaf1fdbc94305b650bd7788675b06
I20260806 12:32:53.116925 25021 tablet_service.cc:1514] Processing DeleteTablet for tablet 562aaf1fdbc94305b650bd7788675b06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:55344
I20260806 12:32:53.116925 24879 tablet_service.cc:1514] Processing DeleteTablet for tablet 562aaf1fdbc94305b650bd7788675b06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:50014
I20260806 12:32:53.117133 24734 tablet_service.cc:1514] Processing DeleteTablet for tablet 562aaf1fdbc94305b650bd7788675b06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:53 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:52636
I20260806 12:32:53.117389 25129 tablet_replica.cc:330] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: stopping tablet replica
I20260806 12:32:53.117720 25129 raft_consensus.cc:2229] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:53.117903 25128 tablet_replica.cc:330] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: stopping tablet replica
I20260806 12:32:53.118079 25129 raft_consensus.cc:2258] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:53.118235 25128 raft_consensus.cc:2229] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:32:53.118470 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24667
I20260806 12:32:53.119745 25129 ts_tablet_manager.cc:1905] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:53.121214 25128 raft_consensus.cc:2258] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:53.122517 25128 ts_tablet_manager.cc:1905] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:53.123099 25129 ts_tablet_manager.cc:1918] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:53.123289 25129 log.cc:1196] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/562aaf1fdbc94305b650bd7788675b06
I20260806 12:32:53.123700 25129 ts_tablet_manager.cc:1939] T 562aaf1fdbc94305b650bd7788675b06 P 431c07b540a94ef8b91237b3ba12a2fe: Deleting consensus metadata
I20260806 12:32:53.124994 24584 catalog_manager.cc:4841] TS 431c07b540a94ef8b91237b3ba12a2fe (127.6.152.131:35785): tablet 562aaf1fdbc94305b650bd7788675b06 (table test-table [id=9b8772841e7347619da99e8f97d842d7]) successfully deleted
I20260806 12:32:53.125276 25128 ts_tablet_manager.cc:1918] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:53.125372 25128 log.cc:1196] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/562aaf1fdbc94305b650bd7788675b06
I20260806 12:32:53.125725 25128 ts_tablet_manager.cc:1939] T 562aaf1fdbc94305b650bd7788675b06 P bfd0548e8221422d821351e6350e01b2: Deleting consensus metadata
I20260806 12:32:53.126737 24583 catalog_manager.cc:4841] TS bfd0548e8221422d821351e6350e01b2 (127.6.152.130:44973): tablet 562aaf1fdbc94305b650bd7788675b06 (table test-table [id=9b8772841e7347619da99e8f97d842d7]) successfully deleted
W20260806 12:32:53.128715 24582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:37529 (error 108)
I20260806 12:32:53.128983 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24807
I20260806 12:32:53.130658 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.130337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:37529 (local address 127.0.0.1:52652)
0806 12:32:53.130481 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.130556 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:37529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 12:32:53.135298 24582 catalog_manager.cc:4584] TS 23999d1943cb4908af42dec716764c28 (127.6.152.129:37529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 562aaf1fdbc94305b650bd7788675b06: Network error: Client connection negotiation failed: client connection to 127.6.152.129:37529: connect: Connection refused (error 111)
I20260806 12:32:53.135936 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24955
I20260806 12:32:53.142725 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 24566
2026-08-06T12:32:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2598 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T12:32:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:32:53Z Disabled control of system clock
I20260806 12:32:53.174552 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35755
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 12:32:53.284664 25135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:53.284909 25135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:53.284943 25135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:53.288827 25135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:53.288874 25135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:53.288892 25135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:53.288909 25135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:53.288924 25135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:53.293547 25135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:53.294564 25135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:53.295953 25135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:53.304327 25142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:53.304354 25144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:53.304447 25135 server_base.cc:1034] running on GCE node
W20260806 12:32:53.304356 25141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:53.305075 25135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:53.306145 25135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:53.307329 25135 hybrid_clock.cc:648] HybridClock initialized: now 1786019573307305 us; error 46 us; skew 500 ppm
I20260806 12:32:53.309139 25135 webserver.cc:460] Webserver started at http://127.6.152.190:38117/ using document root <none> and password file <none>
I20260806 12:32:53.309875 25135 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:53.309974 25135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:53.310277 25135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:53.313197 25135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "97ed37224ab6432c980e54f195a255de"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "50ccd36dbc8078569b5bf27f7591ac95"
server_key_iv: "86cf714c4961189b3f0e8a332ae871d6"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.313920 25135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "97ed37224ab6432c980e54f195a255de"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "50ccd36dbc8078569b5bf27f7591ac95"
server_key_iv: "86cf714c4961189b3f0e8a332ae871d6"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.318714 25135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 12:32:53.321489 25150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.322619 25135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:53.322754 25135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "97ed37224ab6432c980e54f195a255de"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "50ccd36dbc8078569b5bf27f7591ac95"
server_key_iv: "86cf714c4961189b3f0e8a332ae871d6"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.322866 25135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:53.334759 25135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:53.335428 25135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:53.335610 25135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:53.342252 25135 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:35755
I20260806 12:32:53.342283 25204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:35755 every 8 connection(s)
I20260806 12:32:53.343106 25135 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:53.346038 25205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:53.350039 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25135
I20260806 12:32:53.350201 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:32:53.350502 6754 external_mini_cluster.cc:1428] Setting key 7ae6f94796aa527cb171d8555fbb86bf
I20260806 12:32:53.351538 25205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de: Bootstrap starting.
I20260806 12:32:53.354048 25205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:53.354919 25205 log.cc:824] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:53.356822 25205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de: No bootstrap required, opened a new log
I20260806 12:32:53.360116 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.351945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.6.152.190:35755)
0806 12:32:53.352585 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.352607 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.352653 (+ 46us) server_negotiation.cc:407] Connection header received
0806 12:32:53.353434 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.353457 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.353920 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.354292 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:53.354718 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.356081 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.356995 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.357452 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.357629 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.358484 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:53.358489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:53.358501 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:53.358857 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:53.358914 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:53.358934 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:53.359042 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.359210 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.359456 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260806 12:32:53.360611 25205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ed37224ab6432c980e54f195a255de" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35755 } }
I20260806 12:32:53.360867 25205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:53.360913 25205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ed37224ab6432c980e54f195a255de, State: Initialized, Role: FOLLOWER
I20260806 12:32:53.361326 25205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ed37224ab6432c980e54f195a255de" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35755 } }
I20260806 12:32:53.361451 25205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:53.361541 25205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:53.361663 25205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:53.362654 25205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ed37224ab6432c980e54f195a255de" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35755 } }
I20260806 12:32:53.362973 25205 leader_election.cc:302] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97ed37224ab6432c980e54f195a255de; no voters:
I20260806 12:32:53.363323 25205 leader_election.cc:288] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:32:53.363432 25210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:53.363679 25210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [term 1 LEADER]: Becoming Leader. State: Replica: 97ed37224ab6432c980e54f195a255de, State: Running, Role: LEADER
I20260806 12:32:53.364017 25210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ed37224ab6432c980e54f195a255de" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35755 } }
I20260806 12:32:53.364459 25205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:53.365664 25212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97ed37224ab6432c980e54f195a255de. Latest consensus state: current_term: 1 leader_uuid: "97ed37224ab6432c980e54f195a255de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ed37224ab6432c980e54f195a255de" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35755 } } }
I20260806 12:32:53.365797 25212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:53.366245 25221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:53.366160 25211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97ed37224ab6432c980e54f195a255de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ed37224ab6432c980e54f195a255de" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35755 } } }
I20260806 12:32:53.366792 25211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:53.369225 25221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:53.374598 25221 catalog_manager.cc:1347] Generated new cluster ID: 707e2d8921c94ec5a8ad2ef683982f6d
I20260806 12:32:53.374662 25221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:53.385188 25221 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:32:53.386005 25221 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:53.390556 25221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de: Generated new TSK 0
I20260806 12:32:53.391198 25221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:32:53.397781 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35755
--builtin_ntp_servers=127.6.152.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 12:32:53.508602 25231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:53.508827 25231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:53.508867 25231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:53.513080 25231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:53.513131 25231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:53.513217 25231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:53.517737 25231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:53.518754 25231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:53.520113 25231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:53.529680 25240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:53.529796 25231 server_base.cc:1034] running on GCE node
W20260806 12:32:53.529721 25238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:53.529728 25237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:53.530277 25231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:53.530934 25231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:53.532124 25231 hybrid_clock.cc:648] HybridClock initialized: now 1786019573532110 us; error 64 us; skew 500 ppm
I20260806 12:32:53.534044 25231 webserver.cc:460] Webserver started at http://127.6.152.129:34155/ using document root <none> and password file <none>
I20260806 12:32:53.534596 25231 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:53.534686 25231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:53.534917 25231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:53.536972 25231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "6f30c24cd72d4d69a35c597f8d38b94f"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "8d1e4748246d6ed4ca0600aac9c399db"
server_key_iv: "a0537f85a942b378afffb73c5acf1faa"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.537541 25231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "6f30c24cd72d4d69a35c597f8d38b94f"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "8d1e4748246d6ed4ca0600aac9c399db"
server_key_iv: "a0537f85a942b378afffb73c5acf1faa"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.541077 25231 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:32:53.543407 25246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.544582 25231 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:32:53.544714 25231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "6f30c24cd72d4d69a35c597f8d38b94f"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "8d1e4748246d6ed4ca0600aac9c399db"
server_key_iv: "a0537f85a942b378afffb73c5acf1faa"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.544819 25231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:53.553722 25231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:53.554373 25231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:53.554606 25231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:53.555184 25231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:53.556056 25231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:53.556124 25231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.556193 25231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:53.556234 25231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.564869 25231 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:42197
I20260806 12:32:53.564903 25361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:42197 every 8 connection(s)
I20260806 12:32:53.565887 25231 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:53.574554 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25231
I20260806 12:32:53.574698 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:32:53.574831 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.567708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:50687 (local address 127.6.152.190:35755)
0806 12:32:53.567849 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.567853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.568765 (+ 912us) server_negotiation.cc:407] Connection header received
0806 12:32:53.570161 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.570165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.570226 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.570330 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:53.571211 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.571764 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.572868 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.573101 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.573208 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.574337 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:53.574340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:53.574342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:53.574354 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:53.574373 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:53.574381 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:53.574464 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.574629 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.574690 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:32:53.575057 6754 external_mini_cluster.cc:1428] Setting key a7346d620e4744fee02c2a80e3e9b3f1
I20260806 12:32:53.575789 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.568002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35755 (local address 127.6.152.129:50687)
0806 12:32:53.568616 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.568653 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.569946 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.570511 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.570518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.570597 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:53.571038 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.571050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.571897 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.571901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.572737 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.572746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.573257 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.573262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.573421 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.574078 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:32:53.574080 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:53.574117 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:32:53.574175 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:53.574467 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:53.574479 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:53.574487 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.574620 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.574876 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":105,"threads_started":1}
I20260806 12:32:53.576987 25362 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35755
I20260806 12:32:53.577221 25362 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:53.577215 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35755
--builtin_ntp_servers=127.6.152.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 12:32:53.577726 25362 heartbeater.cc:511] Master 127.6.152.190:35755 requested a full tablet report, sending...
I20260806 12:32:53.579092 25167 ts_manager.cc:194] Registered new tserver with Master: 6f30c24cd72d4d69a35c597f8d38b94f (127.6.152.129:42197)
I20260806 12:32:53.580245 25167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.129:50687
W20260806 12:32:53.688045 25366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:53.688283 25366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:53.688319 25366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:53.692116 25366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:53.692154 25366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:53.692224 25366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:53.696631 25366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:53.697506 25366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:53.698758 25366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:53.707985 25376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:53.708011 25374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:53.708065 25373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:53.708191 25366 server_base.cc:1034] running on GCE node
I20260806 12:32:53.708771 25366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:53.709378 25366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:53.710577 25366 hybrid_clock.cc:648] HybridClock initialized: now 1786019573710556 us; error 32 us; skew 500 ppm
I20260806 12:32:53.712375 25366 webserver.cc:460] Webserver started at http://127.6.152.130:41487/ using document root <none> and password file <none>
I20260806 12:32:53.712954 25366 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:53.713045 25366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:53.713279 25366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:53.715394 25366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "830385cafd6a414f8843d0d377728a8a"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "9694c64ad5a8315c75e32bd6b9f99f39"
server_key_iv: "7bc51c70452787ecc879f9190a80fa41"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.715948 25366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "830385cafd6a414f8843d0d377728a8a"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "9694c64ad5a8315c75e32bd6b9f99f39"
server_key_iv: "7bc51c70452787ecc879f9190a80fa41"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.719537 25366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:32:53.721809 25382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.722952 25366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:53.723081 25366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "830385cafd6a414f8843d0d377728a8a"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "9694c64ad5a8315c75e32bd6b9f99f39"
server_key_iv: "7bc51c70452787ecc879f9190a80fa41"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.723242 25366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:53.742339 25366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:53.743002 25366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:53.743220 25366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:53.743804 25366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:53.744742 25366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:53.744814 25366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.744884 25366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:53.744933 25366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.753829 25366 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:33251
I20260806 12:32:53.753863 25498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:33251 every 8 connection(s)
I20260806 12:32:53.754724 25366 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:32:53.763859 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.756460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:53233 (local address 127.6.152.190:35755)
0806 12:32:53.756594 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.756601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.757549 (+ 948us) server_negotiation.cc:407] Connection header received
0806 12:32:53.759001 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.759007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.759072 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.759198 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:53.760048 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.760570 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.761774 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.762048 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.762180 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.763337 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:53.763341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:53.763344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:53.763356 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:53.763380 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:53.763395 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:53.763472 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.763623 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.763682 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:32:53.764566 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25366
I20260806 12:32:53.764714 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:32:53.764882 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.756755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35755 (local address 127.6.152.130:53233)
0806 12:32:53.757398 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.757437 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.758727 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.759364 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.759383 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.759466 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:53.759906 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.759919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.760725 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.760731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.761631 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.761640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.762220 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.762225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.762396 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.763057 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:32:53.763059 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:53.763097 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:32:53.763175 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:53.763474 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:53.763486 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:53.763494 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.763622 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.763924 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
I20260806 12:32:53.765081 6754 external_mini_cluster.cc:1428] Setting key bcbeec60ff821b765fc901fc93d3b513
I20260806 12:32:53.766052 25499 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35755
I20260806 12:32:53.766271 25499 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:53.766830 25499 heartbeater.cc:511] Master 127.6.152.190:35755 requested a full tablet report, sending...
I20260806 12:32:53.767285 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35755
--builtin_ntp_servers=127.6.152.148:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 12:32:53.767887 25167 ts_manager.cc:194] Registered new tserver with Master: 830385cafd6a414f8843d0d377728a8a (127.6.152.130:33251)
I20260806 12:32:53.768451 25167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.130:53233
W20260806 12:32:53.875550 25502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:53.875802 25502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:53.875864 25502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:53.879835 25502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:53.879904 25502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:53.879987 25502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:32:53.884593 25502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:35755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:53.885556 25502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:53.886905 25502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:53.896097 25511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:53.896050 25508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:53.896144 25509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:53.896265 25502 server_base.cc:1034] running on GCE node
I20260806 12:32:53.896701 25502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:53.897286 25502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:53.898468 25502 hybrid_clock.cc:648] HybridClock initialized: now 1786019573898456 us; error 24 us; skew 500 ppm
I20260806 12:32:53.900210 25502 webserver.cc:460] Webserver started at http://127.6.152.131:43463/ using document root <none> and password file <none>
I20260806 12:32:53.900728 25502 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:53.900816 25502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:53.901036 25502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:53.903076 25502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "40fe0acc39714f38bf61bf8f0cfe426a"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "09cd417d40dad9173598c70fcce51bf4"
server_key_iv: "9689f2cb525f2962f4e95f4296f6b17a"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.903702 25502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "40fe0acc39714f38bf61bf8f0cfe426a"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "09cd417d40dad9173598c70fcce51bf4"
server_key_iv: "9689f2cb525f2962f4e95f4296f6b17a"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.906983 25502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:32:53.909317 25518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.910396 25502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:53.910519 25502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "40fe0acc39714f38bf61bf8f0cfe426a"
format_stamp: "Formatted at 2026-08-06 12:32:53 on dist-test-slave-1xrh"
server_key: "09cd417d40dad9173598c70fcce51bf4"
server_key_iv: "9689f2cb525f2962f4e95f4296f6b17a"
server_key_version: "encryptionkey@0"
I20260806 12:32:53.910640 25502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:53.927513 25502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:53.928097 25502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:53.928277 25502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:53.928802 25502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:53.929668 25502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:53.929736 25502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.929802 25502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:53.929848 25502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:53.938850 25502 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:37419
I20260806 12:32:53.938879 25633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:37419 every 8 connection(s)
I20260806 12:32:53.939759 25502 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:32:53.943763 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25502
I20260806 12:32:53.943861 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:32:53.944135 6754 external_mini_cluster.cc:1428] Setting key 23e76b576af0f33d1fb2ed25e6cf31de
I20260806 12:32:53.949842 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.941456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:49667 (local address 127.6.152.190:35755)
0806 12:32:53.941592 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.941600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.942540 (+ 940us) server_negotiation.cc:407] Connection header received
0806 12:32:53.944072 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.944077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.944142 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.944228 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:53.945266 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.945851 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.947470 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.947889 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.947995 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.949298 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:53.949302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:53.949308 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:53.949324 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:53.949364 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:53.949374 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:53.949470 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.949646 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.949706 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:32:53.950623 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.941758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:35755 (local address 127.6.152.131:49667)
0806 12:32:53.942377 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.942432 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.943800 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.944423 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.944435 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.944539 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:53.945110 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.945129 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.945981 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.945985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.947337 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.947353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.948068 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.948075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.948296 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.948980 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:32:53.948983 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:53.949035 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:32:53.949111 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:53.949486 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:53.949496 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:53.949502 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.949642 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.949910 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":107,"threads_started":1}
I20260806 12:32:53.951617 25634 heartbeater.cc:347] Connected to a master server at 127.6.152.190:35755
I20260806 12:32:53.951804 25634 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:53.952237 25634 heartbeater.cc:511] Master 127.6.152.190:35755 requested a full tablet report, sending...
I20260806 12:32:53.953130 25167 ts_manager.cc:194] Registered new tserver with Master: 40fe0acc39714f38bf61bf8f0cfe426a (127.6.152.131:37419)
I20260806 12:32:53.953644 25167 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.131:49667
I20260806 12:32:53.958142 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:32:53.963476 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.960071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.6.152.190:35755)
0806 12:32:53.960223 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.960227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.960423 (+ 196us) server_negotiation.cc:407] Connection header received
0806 12:32:53.960492 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.960494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.960531 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.960592 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:53.960808 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.961323 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.962078 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.962275 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.962346 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.963019 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:53.963022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:53.963024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:53.963034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:53.963051 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:53.963058 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:53.963113 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.963244 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.963289 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 12:32:53.966320 25167 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:32:53.969259 25167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:32:53.982268 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.976786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42197 (local address 127.0.0.1:54200)
0806 12:32:53.977092 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.977147 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.977270 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.978036 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.978041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.978058 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:53.978146 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.978155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.979537 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.979540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.980902 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.980911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.981766 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.981769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.981870 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.981906 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.982001 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.982128 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":43,"thread_start_us":89,"threads_started":1}
I20260806 12:32:53.982278 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.976977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.6.152.129:42197)
0806 12:32:53.977326 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.977341 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.977366 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:32:53.977446 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.977469 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.977673 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.977847 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:53.978263 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.979424 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.981058 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.981641 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.981764 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.981966 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.982025 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.982113 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":126,"threads_started":1}
I20260806 12:32:53.983103 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.977466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:33251 (local address 127.0.0.1:40954)
0806 12:32:53.977802 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.977833 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.977907 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.979174 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.979177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.979194 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:53.979279 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.979284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.980369 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.980375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.981680 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.981687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.982712 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.982716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.982807 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.982831 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.982903 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.982975 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":47,"thread_start_us":92,"threads_started":1}
I20260806 12:32:53.983193 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.977639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.6.152.130:33251)
0806 12:32:53.978718 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.978733 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.978755 (+ 22us) server_negotiation.cc:407] Connection header received
0806 12:32:53.978824 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.978829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.978998 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.979416 (+ 418us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:53.979583 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.980255 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.981813 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.982587 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.982699 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.982864 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.982915 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.983002 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":115,"threads_started":1}
I20260806 12:32:53.983914 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.977801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:37419 (local address 127.0.0.1:41830)
0806 12:32:53.978380 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:53.978393 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:53.978479 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:53.979000 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:53.979004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:53.979021 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:53.979142 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.979154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.980460 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.980464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.981667 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:53.981678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.983498 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.983502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:53.983592 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.983616 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:32:53.983698 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 12:32:53.983765 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":29,"thread_start_us":84,"threads_started":1}
I20260806 12:32:53.984483 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:53.978312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.6.152.131:37419)
0806 12:32:53.978668 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:53.978676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:53.978693 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:32:53.978744 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:53.978748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:53.978872 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:53.979022 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:53.979240 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.980288 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.981812 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:53.982587 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:53.982699 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:53.982864 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:53.984219 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0806 12:32:53.984319 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":94,"threads_started":1}
I20260806 12:32:53.985569 25432 tablet_service.cc:1467] Processing CreateTablet for tablet bfb16aa2d50c4386bfcd95fe80b00772 (DEFAULT_TABLE table=test-table [id=57b31749d86c42b79a1ed94a0422a5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:53.985570 25295 tablet_service.cc:1467] Processing CreateTablet for tablet bfb16aa2d50c4386bfcd95fe80b00772 (DEFAULT_TABLE table=test-table [id=57b31749d86c42b79a1ed94a0422a5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:53.986486 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb16aa2d50c4386bfcd95fe80b00772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:53.986487 25432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb16aa2d50c4386bfcd95fe80b00772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:53.989521 25568 tablet_service.cc:1467] Processing CreateTablet for tablet bfb16aa2d50c4386bfcd95fe80b00772 (DEFAULT_TABLE table=test-table [id=57b31749d86c42b79a1ed94a0422a5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:53.990481 25568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb16aa2d50c4386bfcd95fe80b00772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:53.993155 25651 tablet_bootstrap.cc:492] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Bootstrap starting.
I20260806 12:32:53.995038 25651 tablet_bootstrap.cc:654] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:53.995766 25651 log.cc:824] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:53.996423 25653 tablet_bootstrap.cc:492] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Bootstrap starting.
I20260806 12:32:53.996423 25652 tablet_bootstrap.cc:492] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: Bootstrap starting.
I20260806 12:32:53.998361 25651 tablet_bootstrap.cc:492] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: No bootstrap required, opened a new log
I20260806 12:32:53.998611 25653 tablet_bootstrap.cc:654] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:53.998613 25651 ts_tablet_manager.cc:1397] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:32:53.999447 25653 log.cc:824] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:54.000870 25652 tablet_bootstrap.cc:654] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:54.001379 25653 tablet_bootstrap.cc:492] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: No bootstrap required, opened a new log
I20260806 12:32:54.001551 25653 ts_tablet_manager.cc:1397] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:32:54.001562 25652 log.cc:824] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:54.002506 25651 raft_consensus.cc:348] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.002772 25651 raft_consensus.cc:374] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:54.002861 25651 raft_consensus.cc:729] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830385cafd6a414f8843d0d377728a8a, State: Initialized, Role: FOLLOWER
I20260806 12:32:54.003288 25652 tablet_bootstrap.cc:492] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: No bootstrap required, opened a new log
I20260806 12:32:54.003324 25651 consensus_queue.cc:260] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.003469 25652 ts_tablet_manager.cc:1397] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 12:32:54.004055 25499 heartbeater.cc:503] Master 127.6.152.190:35755 was elected leader, sending a full tablet report...
I20260806 12:32:54.004308 25651 ts_tablet_manager.cc:1428] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 12:32:54.005614 25500 tablet.cc:2369] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:54.005653 25653 raft_consensus.cc:348] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.005860 25653 raft_consensus.cc:374] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:54.005904 25653 raft_consensus.cc:729] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fe0acc39714f38bf61bf8f0cfe426a, State: Initialized, Role: FOLLOWER
I20260806 12:32:54.006347 25653 consensus_queue.cc:260] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.006924 25634 heartbeater.cc:503] Master 127.6.152.190:35755 was elected leader, sending a full tablet report...
I20260806 12:32:54.007272 25653 ts_tablet_manager.cc:1428] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 12:32:54.007766 25652 raft_consensus.cc:348] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.008036 25652 raft_consensus.cc:374] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:54.008140 25652 raft_consensus.cc:729] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f30c24cd72d4d69a35c597f8d38b94f, State: Initialized, Role: FOLLOWER
I20260806 12:32:54.008559 25652 consensus_queue.cc:260] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.009210 25362 heartbeater.cc:503] Master 127.6.152.190:35755 was elected leader, sending a full tablet report...
I20260806 12:32:54.009409 25652 ts_tablet_manager.cc:1428] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 12:32:54.067085 25364 tablet.cc:2369] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:32:54.190779 25635 tablet.cc:2369] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:54.260267 25659 raft_consensus.cc:482] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:54.260522 25659 raft_consensus.cc:504] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.261670 25659 leader_election.cc:288] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f30c24cd72d4d69a35c597f8d38b94f (127.6.152.129:42197), 830385cafd6a414f8843d0d377728a8a (127.6.152.130:33251)
I20260806 12:32:54.266305 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:54.261984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42197 (local address 127.6.152.131:44327)
0806 12:32:54.262131 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:54.262149 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:54.262256 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:54.262680 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:54.262683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:54.262704 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:54.262891 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:54.262902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.263884 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.263888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:54.264946 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:54.264958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.265857 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.265863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:54.265974 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:54.266001 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 12:32:54.266078 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 12:32:54.266148 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 12:32:54.266332 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:54.262094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:44327 (local address 127.6.152.129:42197)
0806 12:32:54.262230 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:54.262251 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:54.262274 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:32:54.262426 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:54.262429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:54.262495 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:54.262597 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:54.263018 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.263690 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.265058 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.265678 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.265822 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:54.265886 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:54.266121 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 12:32:54.266202 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:32:54.266456 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:54.262296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:33251 (local address 127.6.152.131:41133)
0806 12:32:54.262712 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:54.262735 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:54.262817 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:54.263091 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:54.263094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:54.263111 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:54.263253 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:54.263260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.264093 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.264097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:54.265156 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:54.265167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.265993 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.265996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:54.266111 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:54.266137 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:32:54.266210 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 12:32:54.266274 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":158,"threads_started":1}
I20260806 12:32:54.266451 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:54.262429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:41133 (local address 127.6.152.130:33251)
0806 12:32:54.262602 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:54.262606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:54.262821 (+ 215us) server_negotiation.cc:407] Connection header received
0806 12:32:54.262913 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:54.262917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:54.262982 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:54.263054 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:54.263375 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.263973 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.265312 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.265861 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.265976 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:54.266021 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:54.266250 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 12:32:54.266335 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 12:32:54.267344 25452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb16aa2d50c4386bfcd95fe80b00772" candidate_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "830385cafd6a414f8843d0d377728a8a" is_pre_election: true
I20260806 12:32:54.267344 25315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb16aa2d50c4386bfcd95fe80b00772" candidate_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" is_pre_election: true
I20260806 12:32:54.267656 25452 raft_consensus.cc:2454] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40fe0acc39714f38bf61bf8f0cfe426a in term 0.
I20260806 12:32:54.267656 25315 raft_consensus.cc:2454] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40fe0acc39714f38bf61bf8f0cfe426a in term 0.
I20260806 12:32:54.268172 25520 leader_election.cc:302] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40fe0acc39714f38bf61bf8f0cfe426a, 830385cafd6a414f8843d0d377728a8a; no voters:
I20260806 12:32:54.268401 25659 raft_consensus.cc:2781] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:32:54.268486 25659 raft_consensus.cc:482] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:32:54.268518 25659 raft_consensus.cc:3037] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:54.269570 25659 raft_consensus.cc:504] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.269906 25659 leader_election.cc:288] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [CANDIDATE]: Term 1 election: Requested vote from peers 6f30c24cd72d4d69a35c597f8d38b94f (127.6.152.129:42197), 830385cafd6a414f8843d0d377728a8a (127.6.152.130:33251)
I20260806 12:32:54.270282 25315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb16aa2d50c4386bfcd95fe80b00772" candidate_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f30c24cd72d4d69a35c597f8d38b94f"
I20260806 12:32:54.270361 25452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb16aa2d50c4386bfcd95fe80b00772" candidate_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "830385cafd6a414f8843d0d377728a8a"
I20260806 12:32:54.270416 25315 raft_consensus.cc:3037] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:54.270542 25452 raft_consensus.cc:3037] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:54.271580 25315 raft_consensus.cc:2454] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40fe0acc39714f38bf61bf8f0cfe426a in term 1.
I20260806 12:32:54.271730 25452 raft_consensus.cc:2454] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40fe0acc39714f38bf61bf8f0cfe426a in term 1.
I20260806 12:32:54.271934 25520 leader_election.cc:302] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40fe0acc39714f38bf61bf8f0cfe426a, 6f30c24cd72d4d69a35c597f8d38b94f; no voters:
I20260806 12:32:54.272115 25659 raft_consensus.cc:2781] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:54.272358 25659 raft_consensus.cc:686] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 1 LEADER]: Becoming Leader. State: Replica: 40fe0acc39714f38bf61bf8f0cfe426a, State: Running, Role: LEADER
I20260806 12:32:54.272712 25659 consensus_queue.cc:237] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } }
I20260806 12:32:54.275835 25166 catalog_manager.cc:5495] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a reported cstate change: term changed from 0 to 1, leader changed from <none> to 40fe0acc39714f38bf61bf8f0cfe426a (127.6.152.131). New cstate: current_term: 1 leader_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40fe0acc39714f38bf61bf8f0cfe426a" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260806 12:32:54.344132 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:54.340421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.6.152.131:37419)
0806 12:32:54.340555 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:54.340559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:54.340573 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:32:54.340692 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:54.340695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:54.340749 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:54.340834 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:32:54.341051 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.341548 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.342516 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:54.342718 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:54.342792 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:54.343100 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:32:54.343698 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:32:54.343786 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:54.343914 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 12:32:54.343957 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:32:54.350049 25315 raft_consensus.cc:1261] T bfb16aa2d50c4386bfcd95fe80b00772 P 6f30c24cd72d4d69a35c597f8d38b94f [term 1 FOLLOWER]: Refusing update from remote peer 40fe0acc39714f38bf61bf8f0cfe426a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:54.350117 25452 raft_consensus.cc:1261] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 1 FOLLOWER]: Refusing update from remote peer 40fe0acc39714f38bf61bf8f0cfe426a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:54.350803 25659 consensus_queue.cc:1035] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [LEADER]: Connected to new peer: Peer: permanent_uuid: "830385cafd6a414f8843d0d377728a8a" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:54.351012 25662 consensus_queue.cc:1035] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f30c24cd72d4d69a35c597f8d38b94f" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:54.360394 25666 mvcc.cc:205] Tried to move back new op lower bound from 7315536176529895424 to 7315536176223617024. Current Snapshot: MvccSnapshot[applied={T|T < 7315536176529895424}]
I20260806 12:32:54.360586 25667 mvcc.cc:205] Tried to move back new op lower bound from 7315536176529895424 to 7315536176223617024. Current Snapshot: MvccSnapshot[applied={T|T < 7315536176529895424}]
I20260806 12:32:54.366662 25166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:54.366837 25166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:54.369262 25166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97ed37224ab6432c980e54f195a255de: Sending DeleteTablet for 3 replicas of tablet bfb16aa2d50c4386bfcd95fe80b00772
I20260806 12:32:54.369946 25295 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb16aa2d50c4386bfcd95fe80b00772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:54 UTC) from {username='slave'} at 127.0.0.1:54200
I20260806 12:32:54.370195 25432 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb16aa2d50c4386bfcd95fe80b00772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:54 UTC) from {username='slave'} at 127.0.0.1:40954
I20260806 12:32:54.370455 25568 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb16aa2d50c4386bfcd95fe80b00772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:54 UTC) from {username='slave'} at 127.0.0.1:41830
I20260806 12:32:54.370622 25675 tablet_replica.cc:330] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: stopping tablet replica
I20260806 12:32:54.370899 25676 tablet_replica.cc:330] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: stopping tablet replica
I20260806 12:32:54.370929 25675 raft_consensus.cc:2229] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:54.371207 25676 raft_consensus.cc:2229] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:32:54.371279 25675 raft_consensus.cc:2258] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:54.371564 25676 raft_consensus.cc:2258] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:54.371645 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25231
I20260806 12:32:54.372745 25675 ts_tablet_manager.cc:1905] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:54.374142 25676 ts_tablet_manager.cc:1905] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:54.375605 25675 ts_tablet_manager.cc:1918] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:54.375700 25675 log.cc:1196] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/bfb16aa2d50c4386bfcd95fe80b00772
I20260806 12:32:54.376070 25675 ts_tablet_manager.cc:1939] T bfb16aa2d50c4386bfcd95fe80b00772 P 830385cafd6a414f8843d0d377728a8a: Deleting consensus metadata
I20260806 12:32:54.376878 25676 ts_tablet_manager.cc:1918] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:54.377031 25676 log.cc:1196] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/bfb16aa2d50c4386bfcd95fe80b00772
I20260806 12:32:54.377272 25152 catalog_manager.cc:4841] TS 830385cafd6a414f8843d0d377728a8a (127.6.152.130:33251): tablet bfb16aa2d50c4386bfcd95fe80b00772 (table test-table [id=57b31749d86c42b79a1ed94a0422a5b3]) successfully deleted
I20260806 12:32:54.377482 25676 ts_tablet_manager.cc:1939] T bfb16aa2d50c4386bfcd95fe80b00772 P 40fe0acc39714f38bf61bf8f0cfe426a: Deleting consensus metadata
I20260806 12:32:54.378432 25152 catalog_manager.cc:4841] TS 40fe0acc39714f38bf61bf8f0cfe426a (127.6.152.131:37419): tablet bfb16aa2d50c4386bfcd95fe80b00772 (table test-table [id=57b31749d86c42b79a1ed94a0422a5b3]) successfully deleted
W20260806 12:32:54.384140 25152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:42197 (error 108)
I20260806 12:32:54.384397 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25366
I20260806 12:32:54.385846 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:54.385281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42197 (local address 127.0.0.1:54210)
0806 12:32:54.385663 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:54.385744 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:42197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
W20260806 12:32:54.386051 25152 catalog_manager.cc:4584] TS 6f30c24cd72d4d69a35c597f8d38b94f (127.6.152.129:42197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfb16aa2d50c4386bfcd95fe80b00772: Network error: Client connection negotiation failed: client connection to 127.6.152.129:42197: connect: Connection refused (error 111)
I20260806 12:32:54.391672 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25502
I20260806 12:32:54.398233 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25135
2026-08-06T12:32:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1253 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:32:54 dist-test-slave-1xrh krb5kdc[25681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:32:54 dist-test-slave-1xrh krb5kdc[25681](info): set up 2 sockets
Aug 06 12:32:54 dist-test-slave-1xrh krb5kdc[25681](info): commencing operation
krb5kdc: starting...
W20260806 12:32:55.454506 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:32:55 dist-test-slave-1xrh krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:32:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:32:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:32:55.580516 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45533
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:45533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:55.689260 25695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:55.689502 25695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:55.689543 25695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:55.693427 25695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:55.693485 25695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:55.693503 25695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:55.693519 25695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:55.693534 25695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:55.698693 25695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:45533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:55.699662 25695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:55.700640 25695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:55.708735 25704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:55.708793 25695 server_base.cc:1034] running on GCE node
W20260806 12:32:55.708704 25701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:55.708714 25702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:55.709564 25695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:55.710573 25695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:55.711740 25695 hybrid_clock.cc:648] HybridClock initialized: now 1786019575711716 us; error 36 us; skew 500 ppm
Aug 06 12:32:55 dist-test-slave-1xrh krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019575, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:55.721545 25695 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:32:55.722471 25695 webserver.cc:460] Webserver started at http://127.6.152.190:41043/ using document root <none> and password file <none>
I20260806 12:32:55.723057 25695 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:55.723165 25695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:55.723400 25695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:55.725419 25695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "6ae62700cc0b4535b966996f7bd70255"
format_stamp: "Formatted at 2026-08-06 12:32:55 on dist-test-slave-1xrh"
server_key: "1ba902f92c74a972904bc2837fa5617c"
server_key_iv: "390e3818fb76804f1e40e4a7e21955bf"
server_key_version: "encryptionkey@0"
I20260806 12:32:55.726027 25695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "6ae62700cc0b4535b966996f7bd70255"
format_stamp: "Formatted at 2026-08-06 12:32:55 on dist-test-slave-1xrh"
server_key: "1ba902f92c74a972904bc2837fa5617c"
server_key_iv: "390e3818fb76804f1e40e4a7e21955bf"
server_key_version: "encryptionkey@0"
I20260806 12:32:55.729735 25695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 12:32:55.732152 25711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:55.733331 25695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:32:55.733477 25695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "6ae62700cc0b4535b966996f7bd70255"
format_stamp: "Formatted at 2026-08-06 12:32:55 on dist-test-slave-1xrh"
server_key: "1ba902f92c74a972904bc2837fa5617c"
server_key_iv: "390e3818fb76804f1e40e4a7e21955bf"
server_key_version: "encryptionkey@0"
I20260806 12:32:55.733595 25695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:55.760586 25695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:55.763996 25695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:55.764207 25695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:55.771644 25695 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:45533
I20260806 12:32:55.771672 25765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:45533 every 8 connection(s)
I20260806 12:32:55.772614 25695 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:55.775336 25766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:55.776664 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25695
I20260806 12:32:55.776815 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:32:55.777194 6754 external_mini_cluster.cc:1428] Setting key 318328d3065e8358ba61e8a9558f4b56
I20260806 12:32:55.781538 25766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255: Bootstrap starting.
I20260806 12:32:55.783751 25766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:55.784507 25766 log.cc:824] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:32:55 dist-test-slave-1xrh krb5kdc[25681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:55.786818 25766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255: No bootstrap required, opened a new log
I20260806 12:32:55.790807 25766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae62700cc0b4535b966996f7bd70255" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45533 } }
I20260806 12:32:55.791013 25766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:55.791100 25766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae62700cc0b4535b966996f7bd70255, State: Initialized, Role: FOLLOWER
I20260806 12:32:55.791584 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:55.778648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.6.152.190:45533)
0806 12:32:55.779110 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:55.779179 (+ 69us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:55.779224 (+ 45us) server_negotiation.cc:407] Connection header received
0806 12:32:55.779855 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:55.779872 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:55.780187 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:55.780539 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:55.781359 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:55.782364 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:55.783264 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:55.783720 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:55.783917 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:55.786828 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:55.786834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:55.786847 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:55.786898 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:55.788646 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:55.789158 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:55.789163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:55.789168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:55.789275 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:55.789558 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:55.789560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:55.789561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:55.789915 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:55.790066 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:55.790376 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 12:32:55.790721 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":159,"threads_started":1}
I20260806 12:32:55.791625 25766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae62700cc0b4535b966996f7bd70255" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45533 } }
I20260806 12:32:55.791780 25766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:55.791843 25766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:55.791952 25766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:55.792780 25766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae62700cc0b4535b966996f7bd70255" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45533 } }
I20260806 12:32:55.793099 25766 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ae62700cc0b4535b966996f7bd70255; no voters:
I20260806 12:32:55.793421 25766 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:32:55.793485 25771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:55.793685 25771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae62700cc0b4535b966996f7bd70255, State: Running, Role: LEADER
I20260806 12:32:55.793995 25771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae62700cc0b4535b966996f7bd70255" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45533 } }
I20260806 12:32:55.794651 25766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:55.795769 25772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae62700cc0b4535b966996f7bd70255. Latest consensus state: current_term: 1 leader_uuid: "6ae62700cc0b4535b966996f7bd70255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae62700cc0b4535b966996f7bd70255" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45533 } } }
I20260806 12:32:55.795773 25773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae62700cc0b4535b966996f7bd70255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae62700cc0b4535b966996f7bd70255" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45533 } } }
I20260806 12:32:55.795894 25772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:55.795922 25773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:55.796278 25780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:55.799055 25780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:55.804272 25780 catalog_manager.cc:1347] Generated new cluster ID: d7263518581643bc9671edf62388a4cb
I20260806 12:32:55.804332 25780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:55.814245 25780 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:32:55.815084 25780 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:55.819944 25780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255: Generated new TSK 0
I20260806 12:32:55.820566 25780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:32:55.851966 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45533
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:55.967480 25792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:55.967708 25792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:55.967784 25792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:55.971802 25792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:55.971855 25792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:55.971958 25792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:55.977119 25792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:55.978106 25792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:55.979110 25792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:55.988570 25797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:55.988623 25801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:55.988623 25798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:55.988864 25792 server_base.cc:1034] running on GCE node
I20260806 12:32:55.989250 25792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:55.989933 25792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:55.991158 25792 hybrid_clock.cc:648] HybridClock initialized: now 1786019575991109 us; error 34 us; skew 500 ppm
Aug 06 12:32:55 dist-test-slave-1xrh krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019575, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:55.993888 25792 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:32:55.994817 25792 webserver.cc:460] Webserver started at http://127.6.152.129:45759/ using document root <none> and password file <none>
I20260806 12:32:55.995467 25792 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:55.995569 25792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:55.995747 25792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:55.997792 25792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "b1a5c21a047241b98bb7ba44c8042d7a"
format_stamp: "Formatted at 2026-08-06 12:32:55 on dist-test-slave-1xrh"
server_key: "d68d3d7eee2eccd4562ad2b5094e9073"
server_key_iv: "22951fd9746e15e0cd8b279b1c2b0abd"
server_key_version: "encryptionkey@0"
I20260806 12:32:55.998294 25792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "b1a5c21a047241b98bb7ba44c8042d7a"
format_stamp: "Formatted at 2026-08-06 12:32:55 on dist-test-slave-1xrh"
server_key: "d68d3d7eee2eccd4562ad2b5094e9073"
server_key_iv: "22951fd9746e15e0cd8b279b1c2b0abd"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.001850 25792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 12:32:56.004176 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.005318 25792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:56.005443 25792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "b1a5c21a047241b98bb7ba44c8042d7a"
format_stamp: "Formatted at 2026-08-06 12:32:55 on dist-test-slave-1xrh"
server_key: "d68d3d7eee2eccd4562ad2b5094e9073"
server_key_iv: "22951fd9746e15e0cd8b279b1c2b0abd"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.005538 25792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:56.026890 25792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:56.029783 25792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:56.029994 25792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:56.030651 25792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:56.031534 25792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:56.031581 25792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.031651 25792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:56.031682 25792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.041550 25792 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:35513
I20260806 12:32:56.041592 25923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:35513 every 8 connection(s)
I20260806 12:32:56.042428 25792 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:56.047992 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25792
I20260806 12:32:56.048137 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:32:56.048470 6754 external_mini_cluster.cc:1428] Setting key fca71754c404e6fe7c00f89f2364ba59
Aug 06 12:32:56 dist-test-slave-1xrh krb5kdc[25681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019575, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:56.056967 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.044198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:35657 (local address 127.6.152.190:45533)
0806 12:32:56.044366 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.044372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.045239 (+ 867us) server_negotiation.cc:407] Connection header received
0806 12:32:56.046464 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.046467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.046512 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.046589 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:56.047810 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.048627 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.050053 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.050416 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.050539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.053020 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:56.053026 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:56.053029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:56.053062 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:56.054596 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.055307 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.055310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.055311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.055391 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.055793 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.055795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.055796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.055964 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:56.056055 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.056727 (+ 672us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.056833 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:32:56.058148 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.044617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45533 (local address 127.6.152.129:35657)
0806 12:32:56.045087 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.045129 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.046276 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.046754 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.046761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.047175 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:56.047659 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.047672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.048753 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.048756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.049799 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.049812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.050572 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.050574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.050712 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.051387 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:56.051405 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:56.052855 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:56.054765 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:56.054775 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:56.054793 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:56.055211 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:56.055487 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:56.055494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:56.055499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:56.055691 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:56.056098 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:56.056108 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:56.056456 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.056685 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.057034 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:32:56.059504 25924 heartbeater.cc:347] Connected to a master server at 127.6.152.190:45533
I20260806 12:32:56.059713 25924 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:56.060143 25924 heartbeater.cc:511] Master 127.6.152.190:45533 requested a full tablet report, sending...
I20260806 12:32:56.061703 25728 ts_manager.cc:194] Registered new tserver with Master: b1a5c21a047241b98bb7ba44c8042d7a (127.6.152.129:35513)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
I20260806 12:32:56.063341 25728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:35657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:32:56.078517 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45533
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:56.188442 25929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:56.188728 25929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:56.188813 25929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:56.192734 25929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:56.192821 25929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:56.192934 25929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:56.198262 25929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:56.199282 25929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:56.200245 25929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:56.209805 25935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:56.209897 25929 server_base.cc:1034] running on GCE node
W20260806 12:32:56.209802 25937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:56.209916 25934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:56.210415 25929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:56.211097 25929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:56.212318 25929 hybrid_clock.cc:648] HybridClock initialized: now 1786019576212296 us; error 30 us; skew 500 ppm
Aug 06 12:32:56 dist-test-slave-1xrh krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:56.215191 25929 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:32:56.215945 25929 webserver.cc:460] Webserver started at http://127.6.152.130:38955/ using document root <none> and password file <none>
I20260806 12:32:56.216435 25929 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:56.216503 25929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:56.216668 25929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:56.218577 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "0fd4532f5cf04520886887b736d089aa"
format_stamp: "Formatted at 2026-08-06 12:32:56 on dist-test-slave-1xrh"
server_key: "856a55b5aef117cae430731a10857554"
server_key_iv: "aecbb9a037af0953e8e68d2ad9d1b410"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.219086 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "0fd4532f5cf04520886887b736d089aa"
format_stamp: "Formatted at 2026-08-06 12:32:56 on dist-test-slave-1xrh"
server_key: "856a55b5aef117cae430731a10857554"
server_key_iv: "aecbb9a037af0953e8e68d2ad9d1b410"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.222680 25929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:32:56.225064 25944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.226202 25929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:56.226293 25929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "0fd4532f5cf04520886887b736d089aa"
format_stamp: "Formatted at 2026-08-06 12:32:56 on dist-test-slave-1xrh"
server_key: "856a55b5aef117cae430731a10857554"
server_key_iv: "aecbb9a037af0953e8e68d2ad9d1b410"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.226372 25929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:56.244724 25929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:56.247942 25929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:56.248093 25929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:56.248590 25929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:56.249322 25929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:56.249368 25929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.249409 25929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:56.249428 25929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.258236 25929 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:41979
I20260806 12:32:56.258258 26060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:41979 every 8 connection(s)
I20260806 12:32:56.259078 25929 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:32:56.265055 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 25929
I20260806 12:32:56.265213 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:32:56.265534 6754 external_mini_cluster.cc:1428] Setting key af407f9f84db3de0ce1a59303aaf5f7e
Aug 06 12:32:56 dist-test-slave-1xrh krb5kdc[25681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:56.273051 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.260844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:59109 (local address 127.6.152.190:45533)
0806 12:32:56.261003 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.261008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.261804 (+ 796us) server_negotiation.cc:407] Connection header received
0806 12:32:56.262994 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.262996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.263041 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.263114 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:56.264258 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.264794 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.266347 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.266585 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.266699 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.269281 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:56.269287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:56.269292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:56.269323 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:56.270994 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.271525 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.271532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.271537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.271630 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.271970 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.271976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.271979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.272240 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:56.272356 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.272833 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.272937 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 12:32:56.274169 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.261248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45533 (local address 127.6.152.130:59109)
0806 12:32:56.261658 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.261698 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.262804 (+ 1106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.263323 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.263329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.263678 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:56.264133 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.264146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.264921 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.264927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.266241 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.266254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.266736 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.266741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.266913 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.267512 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:56.267538 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:56.269135 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:56.271118 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:56.271164 (+ 46us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:56.271181 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:56.271424 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:56.271730 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:56.271732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:56.271734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:56.271861 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:56.272346 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:56.272351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:56.272552 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.272740 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.273086 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":105,"threads_started":1}
I20260806 12:32:56.275348 26061 heartbeater.cc:347] Connected to a master server at 127.6.152.190:45533
I20260806 12:32:56.275591 26061 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:56.276033 26061 heartbeater.cc:511] Master 127.6.152.190:45533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
I20260806 12:32:56.277067 25728 ts_manager.cc:194] Registered new tserver with Master: 0fd4532f5cf04520886887b736d089aa (127.6.152.130:41979)
I20260806 12:32:56.277592 25728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:59109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:32:56.295061 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45533
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:56.405148 26066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:56.405382 26066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:56.405424 26066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:56.409698 26066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:56.409751 26066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:56.409807 26066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:32:56.415042 26066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:45533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:56.416038 26066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:56.417039 26066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:56.426820 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:56.426908 26066 server_base.cc:1034] running on GCE node
W20260806 12:32:56.426853 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:56.426843 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:56.427444 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:56.428016 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:56.429199 26066 hybrid_clock.cc:648] HybridClock initialized: now 1786019576429188 us; error 22 us; skew 500 ppm
Aug 06 12:32:56 dist-test-slave-1xrh krb5kdc[25681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:56.431973 26066 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:32:56.432766 26066 webserver.cc:460] Webserver started at http://127.6.152.131:34093/ using document root <none> and password file <none>
I20260806 12:32:56.433329 26066 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:56.433423 26066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:56.433640 26066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:56.435670 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "7130ca2b2cf64e2f9ee8f32d7b872381"
format_stamp: "Formatted at 2026-08-06 12:32:56 on dist-test-slave-1xrh"
server_key: "c51710422c88e6157c0bb2a9b58e0d6b"
server_key_iv: "4e7a3872f98a2477894765cdf26fe25e"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.436250 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "7130ca2b2cf64e2f9ee8f32d7b872381"
format_stamp: "Formatted at 2026-08-06 12:32:56 on dist-test-slave-1xrh"
server_key: "c51710422c88e6157c0bb2a9b58e0d6b"
server_key_iv: "4e7a3872f98a2477894765cdf26fe25e"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.439935 26066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:32:56.442291 26082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.443522 26066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:32:56.443647 26066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "7130ca2b2cf64e2f9ee8f32d7b872381"
format_stamp: "Formatted at 2026-08-06 12:32:56 on dist-test-slave-1xrh"
server_key: "c51710422c88e6157c0bb2a9b58e0d6b"
server_key_iv: "4e7a3872f98a2477894765cdf26fe25e"
server_key_version: "encryptionkey@0"
I20260806 12:32:56.443749 26066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:56.469321 26066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:56.472633 26066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:56.472826 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:56.473414 26066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:56.474198 26066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:56.474270 26066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.474341 26066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:56.474396 26066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:56.483453 26066 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:36657
I20260806 12:32:56.483495 26197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:36657 every 8 connection(s)
I20260806 12:32:56.484308 26066 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:32:56.491357 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26066
I20260806 12:32:56.491557 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:32:56.491870 6754 external_mini_cluster.cc:1428] Setting key ef3d3a6806a2cc3f562198839fa42741
Aug 06 12:32:56 dist-test-slave-1xrh krb5kdc[25681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019576, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:56.497977 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.486001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:48565 (local address 127.6.152.190:45533)
0806 12:32:56.486136 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.486140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.486965 (+ 825us) server_negotiation.cc:407] Connection header received
0806 12:32:56.488251 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.488255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.488314 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.488433 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:56.489564 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.490103 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.491295 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.491617 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.491722 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.494314 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:56.494318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:56.494321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:56.494355 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:56.495874 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.496480 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.496484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.496486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.496578 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.496910 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.496914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.496915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.497094 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:56.497200 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.497663 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.497800 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:32:56.498719 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.486372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:45533 (local address 127.6.152.131:48565)
0806 12:32:56.486815 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.486857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.488039 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.488606 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.488613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.488950 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:56.489403 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.489417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.490235 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.490238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.491111 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.491122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.491762 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.491766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.491935 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.492568 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:56.492584 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:56.494153 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:56.496007 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:56.496012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:56.496027 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:56.496373 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:56.496669 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:56.496672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:56.496674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:56.496810 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:56.497199 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:56.497206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:56.497444 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.497622 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.497945 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":113,"threads_started":1}
I20260806 12:32:56.499785 26198 heartbeater.cc:347] Connected to a master server at 127.6.152.190:45533
I20260806 12:32:56.499994 26198 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:56.500413 26198 heartbeater.cc:511] Master 127.6.152.190:45533 requested a full tablet report, sending...
I20260806 12:32:56.501396 25728 ts_manager.cc:194] Registered new tserver with Master: 7130ca2b2cf64e2f9ee8f32d7b872381 (127.6.152.131:36657)
I20260806 12:32:56.501935 25728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:48565
I20260806 12:32:56.505769 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:32:56.515332 25769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.507765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.6.152.190:45533)
0806 12:32:56.507979 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.507983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.508135 (+ 152us) server_negotiation.cc:407] Connection header received
0806 12:32:56.508200 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.508202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.508241 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.508333 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:56.509004 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.509562 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.510538 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.510730 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.510857 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.512336 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:56.512339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:56.512340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:56.512362 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:56.513537 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.514007 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.514009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.514010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.514080 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:56.514351 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:56.514353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:56.514355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:56.514619 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:56.514742 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.515036 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.515116 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260806 12:32:56.518257 25728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:32:56.521279 25728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:32:56.535421 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.529388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:41979 (local address 127.0.0.1:56562)
0806 12:32:56.530175 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.530229 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.530369 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.531191 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.531197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.531222 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.531367 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.531374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.532513 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.532518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.533706 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.533717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.534844 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.534849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.534961 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.535003 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.535141 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.535265 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":3,"thread_start_us":75,"threads_started":1}
I20260806 12:32:56.535462 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.529522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.6.152.130:41979)
0806 12:32:56.530115 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.530130 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.530303 (+ 173us) server_negotiation.cc:407] Connection header received
0806 12:32:56.530579 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.530585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.530794 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.530961 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.531502 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.532380 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.533847 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.534712 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.534817 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.534961 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.535187 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.535291 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":98,"threads_started":1}
I20260806 12:32:56.535791 26211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.529836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36657 (local address 127.0.0.1:46370)
0806 12:32:56.530998 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.531010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.531093 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.531653 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.531655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.531674 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.531760 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.531766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.533152 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.533155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.534285 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.534294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.535406 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.535409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.535506 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.535541 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.535619 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.535674 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"mutex_wait_us":66,"thread_start_us":95,"threads_started":1}
I20260806 12:32:56.536047 26212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.529528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:35513 (local address 127.0.0.1:35664)
0806 12:32:56.530664 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.530681 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.530775 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.531489 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.531492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.531510 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.531621 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.531627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.533120 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.533123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.534311 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.534321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.535615 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.535619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.535712 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.535732 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.535825 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.535892 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"mutex_wait_us":28,"thread_start_us":90,"threads_started":1}
I20260806 12:32:56.536140 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.530216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.6.152.129:35513)
0806 12:32:56.530542 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.530557 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.530869 (+ 312us) server_negotiation.cc:407] Connection header received
0806 12:32:56.530954 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.530959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.531172 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.531316 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.531747 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.532976 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.534477 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.535300 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.535398 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.535543 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.535858 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.535959 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":127,"threads_started":1}
I20260806 12:32:56.536346 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.530216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.6.152.131:36657)
0806 12:32:56.530863 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.530876 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.531203 (+ 327us) server_negotiation.cc:407] Connection header received
0806 12:32:56.531347 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.531351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.531516 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.531660 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.531879 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.532978 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.534412 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.535108 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.535224 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.535345 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.535967 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.536108 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":143,"threads_started":1}
I20260806 12:32:56.539081 26131 tablet_service.cc:1467] Processing CreateTablet for tablet 282083d942e64e00aec19cc3629140a3 (DEFAULT_TABLE table=test-table [id=7ce177031f2d450eab418c95b6a191d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:56.539083 25995 tablet_service.cc:1467] Processing CreateTablet for tablet 282083d942e64e00aec19cc3629140a3 (DEFAULT_TABLE table=test-table [id=7ce177031f2d450eab418c95b6a191d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:56.540241 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 282083d942e64e00aec19cc3629140a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:56.540237 25995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 282083d942e64e00aec19cc3629140a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:56.544895 25858 tablet_service.cc:1467] Processing CreateTablet for tablet 282083d942e64e00aec19cc3629140a3 (DEFAULT_TABLE table=test-table [id=7ce177031f2d450eab418c95b6a191d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:56.545960 25858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 282083d942e64e00aec19cc3629140a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:56.547889 26216 tablet_bootstrap.cc:492] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Bootstrap starting.
I20260806 12:32:56.548135 26217 tablet_bootstrap.cc:492] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Bootstrap starting.
I20260806 12:32:56.550524 26216 tablet_bootstrap.cc:654] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:56.552524 26217 tablet_bootstrap.cc:654] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:56.552609 26216 log.cc:824] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:56.553035 26219 tablet_bootstrap.cc:492] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: Bootstrap starting.
I20260806 12:32:56.553339 26217 log.cc:824] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:56.554864 26216 tablet_bootstrap.cc:492] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: No bootstrap required, opened a new log
I20260806 12:32:56.554984 26219 tablet_bootstrap.cc:654] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:56.555117 26216 ts_tablet_manager.cc:1397] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 12:32:56.555428 26217 tablet_bootstrap.cc:492] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: No bootstrap required, opened a new log
I20260806 12:32:56.555644 26217 ts_tablet_manager.cc:1397] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 12:32:56.556232 26219 log.cc:824] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:56.558110 26219 tablet_bootstrap.cc:492] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: No bootstrap required, opened a new log
I20260806 12:32:56.558346 26219 ts_tablet_manager.cc:1397] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260806 12:32:56.558730 26216 raft_consensus.cc:348] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.558979 26216 raft_consensus.cc:374] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:56.559052 26216 raft_consensus.cc:729] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7130ca2b2cf64e2f9ee8f32d7b872381, State: Initialized, Role: FOLLOWER
I20260806 12:32:56.559307 26217 raft_consensus.cc:348] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.559562 26217 raft_consensus.cc:374] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:56.559532 26216 consensus_queue.cc:260] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.559659 26217 raft_consensus.cc:729] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd4532f5cf04520886887b736d089aa, State: Initialized, Role: FOLLOWER
I20260806 12:32:56.560093 26217 consensus_queue.cc:260] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.560189 26198 heartbeater.cc:503] Master 127.6.152.190:45533 was elected leader, sending a full tablet report...
I20260806 12:32:56.560703 26061 heartbeater.cc:503] Master 127.6.152.190:45533 was elected leader, sending a full tablet report...
I20260806 12:32:56.561048 26217 ts_tablet_manager.cc:1428] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 12:32:56.561852 26216 ts_tablet_manager.cc:1428] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 12:32:56.562278 26219 raft_consensus.cc:348] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.562489 26219 raft_consensus.cc:374] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:56.562566 26219 raft_consensus.cc:729] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a5c21a047241b98bb7ba44c8042d7a, State: Initialized, Role: FOLLOWER
I20260806 12:32:56.562973 26219 consensus_queue.cc:260] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.563547 25924 heartbeater.cc:503] Master 127.6.152.190:45533 was elected leader, sending a full tablet report...
I20260806 12:32:56.563881 26219 ts_tablet_manager.cc:1428] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 12:32:56.735281 26199 tablet.cc:2369] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:32:56.760293 26062 tablet.cc:2369] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:32:56.793802 25925 tablet.cc:2369] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:56.940194 26224 raft_consensus.cc:482] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:56.940511 26223 raft_consensus.cc:482] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:56.940461 26224 raft_consensus.cc:504] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.940676 26223 raft_consensus.cc:504] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.941910 26223 leader_election.cc:288] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd4532f5cf04520886887b736d089aa (127.6.152.130:41979), b1a5c21a047241b98bb7ba44c8042d7a (127.6.152.129:35513)
I20260806 12:32:56.941910 26224 leader_election.cc:288] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1a5c21a047241b98bb7ba44c8042d7a (127.6.152.129:35513), 7130ca2b2cf64e2f9ee8f32d7b872381 (127.6.152.131:36657)
I20260806 12:32:56.942902 26225 raft_consensus.cc:482] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:56.943100 26225 raft_consensus.cc:504] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.944335 26225 leader_election.cc:288] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd4532f5cf04520886887b736d089aa (127.6.152.130:41979), 7130ca2b2cf64e2f9ee8f32d7b872381 (127.6.152.131:36657)
I20260806 12:32:56.947310 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:35513 (local address 127.6.152.131:59283)
0806 12:32:56.942384 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.942402 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.942511 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.943221 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.943227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.943251 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.943381 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.943389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.944497 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.944500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.945754 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.945766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.946837 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.946842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.946976 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.947002 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.947073 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.947173 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 12:32:56.947921 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:59283 (local address 127.6.152.129:35513)
0806 12:32:56.942840 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.942845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.942859 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:32:56.942934 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.942938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.942991 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.943070 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.943508 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.944336 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.945903 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.946570 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.946716 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.946772 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.947736 (+ 964us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.947808 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":55}
I20260806 12:32:56.948474 25878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" is_pre_election: true
I20260806 12:32:56.948694 25878 raft_consensus.cc:2454] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in term 0.
I20260806 12:32:56.948781 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36657 (local address 127.6.152.130:35835)
0806 12:32:56.943264 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.943277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.943454 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.943812 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.943815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.943831 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.943965 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.943971 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.944988 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.944992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.947386 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.947397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.948331 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.948335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.948437 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.948473 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.948551 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.948624 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":1030,"spinlock_wait_cycles":39552,"thread_start_us":85,"threads_started":1}
I20260806 12:32:56.949200 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:35835 (local address 127.6.152.131:36657)
0806 12:32:56.942639 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.942646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.943341 (+ 695us) server_negotiation.cc:407] Connection header received
0806 12:32:56.943574 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.943577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.943619 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.943664 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.944098 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.944836 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.947560 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.948103 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.948212 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.948266 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.948931 (+ 665us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.949082 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":39}
I20260806 12:32:56.949477 26084 leader_election.cc:302] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7130ca2b2cf64e2f9ee8f32d7b872381, b1a5c21a047241b98bb7ba44c8042d7a; no voters:
I20260806 12:32:56.949662 26152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "0fd4532f5cf04520886887b736d089aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" is_pre_election: true
I20260806 12:32:56.949745 26223 raft_consensus.cc:2781] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:32:56.949862 26152 raft_consensus.cc:2454] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd4532f5cf04520886887b736d089aa in term 0.
I20260806 12:32:56.950019 26223 raft_consensus.cc:482] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:32:56.950095 26223 raft_consensus.cc:3037] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:56.950297 25945 leader_election.cc:302] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd4532f5cf04520886887b736d089aa, 7130ca2b2cf64e2f9ee8f32d7b872381; no voters:
I20260806 12:32:56.951067 26227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:35513 (local address 127.6.152.130:42653)
0806 12:32:56.942594 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.942608 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.942721 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.943370 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.943373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.943437 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.943564 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.943571 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.944620 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.944623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.948500 (+ 3877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.948510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.950653 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.950657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.950758 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.950781 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.950843 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.950918 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":604,"spinlock_wait_cycles":26368,"thread_start_us":97,"threads_started":1}
I20260806 12:32:56.951440 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:42653 (local address 127.6.152.129:35513)
0806 12:32:56.943070 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.943078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.943090 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:32:56.943148 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.943150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.943186 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.943261 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.943698 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.944437 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.948631 (+ 4194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.949261 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.949380 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.949423 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.951239 (+ 1816us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.951310 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
I20260806 12:32:56.951443 26223 raft_consensus.cc:504] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.951716 26230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.944658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36657 (local address 127.6.152.129:54129)
0806 12:32:56.945278 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.945292 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.945385 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.945819 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.945822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.945859 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.946337 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.946347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.947432 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.947435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.948660 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.948670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.951296 (+ 2626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.951299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.951416 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.951442 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.951511 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.951582 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":307,"thread_start_us":107,"threads_started":1}
I20260806 12:32:56.951784 26223 leader_election.cc:288] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [CANDIDATE]: Term 1 election: Requested vote from peers 0fd4532f5cf04520886887b736d089aa (127.6.152.130:41979), b1a5c21a047241b98bb7ba44c8042d7a (127.6.152.129:35513)
I20260806 12:32:56.951912 26224 raft_consensus.cc:2781] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:32:56.952013 26224 raft_consensus.cc:482] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:32:56.952060 26224 raft_consensus.cc:3037] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:56.952140 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.944992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:54129 (local address 127.6.152.131:36657)
0806 12:32:56.945532 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.945542 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.945553 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:32:56.945592 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.945595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.945636 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.945713 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.946473 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.947298 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.948781 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.951033 (+ 2252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.951865 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.951912 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.951947 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.952018 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":112,"threads_started":1}
I20260806 12:32:56.952656 25877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a5c21a047241b98bb7ba44c8042d7a"
I20260806 12:32:56.952778 25877 raft_consensus.cc:3037] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:56.952845 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.943018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:34959 (local address 127.6.152.130:41979)
0806 12:32:56.943150 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.943156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.946261 (+ 3105us) server_negotiation.cc:407] Connection header received
0806 12:32:56.946454 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.946459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.946507 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.946562 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.947111 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.947837 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.951617 (+ 3780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.952251 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.952345 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.952390 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.952676 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.952742 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":451,"server-negotiator.queue_time_us":40}
I20260806 12:32:56.953181 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.942670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:41979 (local address 127.6.152.131:34959)
0806 12:32:56.946139 (+ 3469us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.946153 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.946333 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.946648 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.946651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.946667 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.946784 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.946789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.949473 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.949477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.950521 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.950534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.952370 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.952373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.952504 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.952538 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.952953 (+ 415us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.953037 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3408,"thread_start_us":85,"threads_started":1}
I20260806 12:32:56.953198 25878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "0fd4532f5cf04520886887b736d089aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" is_pre_election: true
I20260806 12:32:56.953413 26224 raft_consensus.cc:504] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.953765 26224 leader_election.cc:288] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [CANDIDATE]: Term 1 election: Requested vote from peers b1a5c21a047241b98bb7ba44c8042d7a (127.6.152.129:35513), 7130ca2b2cf64e2f9ee8f32d7b872381 (127.6.152.131:36657)
I20260806 12:32:56.954079 25877 raft_consensus.cc:2454] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in term 1.
I20260806 12:32:56.954080 26015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd4532f5cf04520886887b736d089aa" is_pre_election: true
I20260806 12:32:56.954295 26015 raft_consensus.cc:2379] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in current term 1: Already voted for candidate 0fd4532f5cf04520886887b736d089aa in this term.
I20260806 12:32:56.954385 26084 leader_election.cc:302] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7130ca2b2cf64e2f9ee8f32d7b872381, b1a5c21a047241b98bb7ba44c8042d7a; no voters:
I20260806 12:32:56.954452 26152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" is_pre_election: true
I20260806 12:32:56.954599 25877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "0fd4532f5cf04520886887b736d089aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1a5c21a047241b98bb7ba44c8042d7a"
I20260806 12:32:56.954639 26152 raft_consensus.cc:2379] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1a5c21a047241b98bb7ba44c8042d7a in current term 1: Already voted for candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in this term.
I20260806 12:32:56.954747 25877 raft_consensus.cc:2379] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fd4532f5cf04520886887b736d089aa in current term 1: Already voted for candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in this term.
I20260806 12:32:56.954802 26014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd4532f5cf04520886887b736d089aa"
I20260806 12:32:56.954932 26014 raft_consensus.cc:2379] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in current term 1: Already voted for candidate 0fd4532f5cf04520886887b736d089aa in this term.
I20260806 12:32:56.955178 26232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.945791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:41979 (local address 127.6.152.129:42633)
0806 12:32:56.946116 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:56.946130 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:56.946211 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:56.946742 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:56.946745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:56.946762 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:56.946884 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.946890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.949953 (+ 3063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.949957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.951040 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:56.951052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.954722 (+ 3670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.954725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:56.954827 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.954850 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 12:32:56.954920 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 12:32:56.954994 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260806 12:32:56.955265 26152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "0fd4532f5cf04520886887b736d089aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381"
I20260806 12:32:56.955387 26152 raft_consensus.cc:2379] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fd4532f5cf04520886887b736d089aa in current term 1: Already voted for candidate 7130ca2b2cf64e2f9ee8f32d7b872381 in this term.
I20260806 12:32:56.955711 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.946118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:42633 (local address 127.6.152.130:41979)
0806 12:32:56.946505 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.946513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.946527 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:32:56.946569 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.946571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.946624 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.946987 (+ 363us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:56.949121 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.949827 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.951216 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.954575 (+ 3359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.955456 (+ 881us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.955503 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.955535 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.955598 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2457,"server-negotiator.queue_time_us":256,"thread_start_us":97,"threads_started":1}
I20260806 12:32:56.955735 26223 raft_consensus.cc:2781] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:56.955700 25945 leader_election.cc:302] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fd4532f5cf04520886887b736d089aa; no voters: 7130ca2b2cf64e2f9ee8f32d7b872381, b1a5c21a047241b98bb7ba44c8042d7a
I20260806 12:32:56.955931 26224 raft_consensus.cc:2726] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 12:32:56.955948 26223 raft_consensus.cc:686] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 LEADER]: Becoming Leader. State: Replica: 7130ca2b2cf64e2f9ee8f32d7b872381, State: Running, Role: LEADER
I20260806 12:32:56.956197 26014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282083d942e64e00aec19cc3629140a3" candidate_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd4532f5cf04520886887b736d089aa" is_pre_election: true
I20260806 12:32:56.956332 26014 raft_consensus.cc:2379] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1a5c21a047241b98bb7ba44c8042d7a in current term 1: Already voted for candidate 0fd4532f5cf04520886887b736d089aa in this term.
I20260806 12:32:56.956305 26223 consensus_queue.cc:237] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } }
I20260806 12:32:56.956597 25812 leader_election.cc:302] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1a5c21a047241b98bb7ba44c8042d7a; no voters: 0fd4532f5cf04520886887b736d089aa, 7130ca2b2cf64e2f9ee8f32d7b872381
I20260806 12:32:56.956815 26225 raft_consensus.cc:2726] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 12:32:56.960630 25723 catalog_manager.cc:5495] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7130ca2b2cf64e2f9ee8f32d7b872381 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7130ca2b2cf64e2f9ee8f32d7b872381" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36657 } health_report { overall_health: HEALTHY } } }
I20260806 12:32:56.986124 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:56.982457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.6.152.131:36657)
0806 12:32:56.982607 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:56.982612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:56.982628 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:32:56.982716 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:56.982718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:56.982772 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:56.982883 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:32:56.983193 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.983712 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.984565 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:56.984784 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:56.984897 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:56.985203 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:32:56.985609 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:32:56.985770 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:56.985897 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 12:32:56.985961 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 12:32:56.991864 25877 raft_consensus.cc:1261] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 1 FOLLOWER]: Refusing update from remote peer 7130ca2b2cf64e2f9ee8f32d7b872381: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:56.991869 26014 raft_consensus.cc:1261] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Refusing update from remote peer 7130ca2b2cf64e2f9ee8f32d7b872381: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:56.992522 26234 consensus_queue.cc:1035] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fd4532f5cf04520886887b736d089aa" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 41979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:56.992697 26223 consensus_queue.cc:1035] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a5c21a047241b98bb7ba44c8042d7a" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 35513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:57.000917 26238 mvcc.cc:205] Tried to move back new op lower bound from 7315536187351318528 to 7315536187216957440. Current Snapshot: MvccSnapshot[applied={T|T < 7315536187351318528}]
I20260806 12:32:57.010532 25723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:57.010666 25723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:57.013000 25723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ae62700cc0b4535b966996f7bd70255: Sending DeleteTablet for 3 replicas of tablet 282083d942e64e00aec19cc3629140a3
I20260806 12:32:57.013746 25858 tablet_service.cc:1514] Processing DeleteTablet for tablet 282083d942e64e00aec19cc3629140a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:57 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:35664
I20260806 12:32:57.013738 26131 tablet_service.cc:1514] Processing DeleteTablet for tablet 282083d942e64e00aec19cc3629140a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:57 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:46370
I20260806 12:32:57.013759 25995 tablet_service.cc:1514] Processing DeleteTablet for tablet 282083d942e64e00aec19cc3629140a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:57 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:56562
I20260806 12:32:57.014249 26248 tablet_replica.cc:330] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a: stopping tablet replica
I20260806 12:32:57.014454 26249 tablet_replica.cc:330] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: stopping tablet replica
I20260806 12:32:57.014624 26248 raft_consensus.cc:2229] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:57.014909 26248 raft_consensus.cc:2258] T 282083d942e64e00aec19cc3629140a3 P b1a5c21a047241b98bb7ba44c8042d7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:57.015522 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25792
I20260806 12:32:57.025077 26247 tablet_replica.cc:330] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: stopping tablet replica
I20260806 12:32:57.025384 26247 raft_consensus.cc:2229] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:57.025707 26247 raft_consensus.cc:2258] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:57.026080 26249 raft_consensus.cc:2229] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:32:57.026432 26249 raft_consensus.cc:2258] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:57.027217 26247 ts_tablet_manager.cc:1905] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:57.027861 26249 ts_tablet_manager.cc:1905] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:57.030664 26249 ts_tablet_manager.cc:1918] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:57.030819 26249 log.cc:1196] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/282083d942e64e00aec19cc3629140a3
I20260806 12:32:57.030916 26247 ts_tablet_manager.cc:1918] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:57.031129 26247 log.cc:1196] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/282083d942e64e00aec19cc3629140a3
I20260806 12:32:57.031291 26249 ts_tablet_manager.cc:1939] T 282083d942e64e00aec19cc3629140a3 P 7130ca2b2cf64e2f9ee8f32d7b872381: Deleting consensus metadata
I20260806 12:32:57.031545 26247 ts_tablet_manager.cc:1939] T 282083d942e64e00aec19cc3629140a3 P 0fd4532f5cf04520886887b736d089aa: Deleting consensus metadata
W20260806 12:32:57.032361 25713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:35513 (error 108)
I20260806 12:32:57.032688 25712 catalog_manager.cc:4841] TS 7130ca2b2cf64e2f9ee8f32d7b872381 (127.6.152.131:36657): tablet 282083d942e64e00aec19cc3629140a3 (table test-table [id=7ce177031f2d450eab418c95b6a191d9]) successfully deleted
I20260806 12:32:57.032816 25715 catalog_manager.cc:4841] TS 0fd4532f5cf04520886887b736d089aa (127.6.152.130:41979): tablet 282083d942e64e00aec19cc3629140a3 (table test-table [id=7ce177031f2d450eab418c95b6a191d9]) successfully deleted
I20260806 12:32:57.033247 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25929
I20260806 12:32:57.034278 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.033432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:35513 (local address 127.0.0.1:35672)
0806 12:32:57.034076 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.034172 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:35513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538}
W20260806 12:32:57.035298 25713 catalog_manager.cc:4584] TS b1a5c21a047241b98bb7ba44c8042d7a (127.6.152.129:35513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 282083d942e64e00aec19cc3629140a3: Network error: Client connection negotiation failed: client connection to 127.6.152.129:35513: connect: Connection refused (error 111)
I20260806 12:32:57.040222 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26066
I20260806 12:32:57.046973 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 25695
2026-08-06T12:32:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2650 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T12:32:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:32:57Z Disabled control of system clock
I20260806 12:32:57.078774 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39961
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:39961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 12:32:57.187238 26254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:57.187480 26254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:57.187515 26254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:57.191324 26254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:57.191367 26254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:57.191383 26254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:57.191406 26254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:57.191421 26254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:57.196007 26254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:39961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:39961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:57.196910 26254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:57.198148 26254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:57.206481 26260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:57.206564 26261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:57.206508 26263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:57.206732 26254 server_base.cc:1034] running on GCE node
I20260806 12:32:57.207378 26254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:57.208680 26254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:57.209900 26254 hybrid_clock.cc:648] HybridClock initialized: now 1786019577209873 us; error 40 us; skew 500 ppm
I20260806 12:32:57.211686 26254 webserver.cc:460] Webserver started at http://127.6.152.190:36051/ using document root <none> and password file <none>
I20260806 12:32:57.212237 26254 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:57.212332 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:57.212555 26254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:57.214617 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "3bb127e62a42b96ca9d1aae7693d9323"
server_key_iv: "366d489931af5b3adadb78e1a5e79ba0"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.215237 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "3bb127e62a42b96ca9d1aae7693d9323"
server_key_iv: "366d489931af5b3adadb78e1a5e79ba0"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.218700 26254 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:32:57.221135 26269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.222234 26254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:57.222371 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "3bb127e62a42b96ca9d1aae7693d9323"
server_key_iv: "366d489931af5b3adadb78e1a5e79ba0"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.222474 26254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:57.234735 26254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:57.235428 26254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:57.235617 26254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:57.242504 26254 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:39961
I20260806 12:32:57.242512 26322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:39961 every 8 connection(s)
I20260806 12:32:57.243425 26254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:57.244005 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26254
I20260806 12:32:57.244161 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:32:57.244458 6754 external_mini_cluster.cc:1428] Setting key 119b0dcc0068934683fb80cd4317b909
I20260806 12:32:57.247465 26323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:57.252712 26323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d: Bootstrap starting.
I20260806 12:32:57.253089 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.245694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60418 (local address 127.6.152.190:39961)
0806 12:32:57.246270 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.246290 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.246343 (+ 53us) server_negotiation.cc:407] Connection header received
0806 12:32:57.246961 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.246977 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.247423 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.247775 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:57.248188 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.249110 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.249987 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.250440 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.250601 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.251506 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:57.251511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:57.251522 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:57.251872 (+ 350us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:57.251924 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:57.251943 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:57.252050 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.252164 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.252411 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":128,"threads_started":1}
I20260806 12:32:57.254798 26323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:57.255550 26323 log.cc:824] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:57.257283 26323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d: No bootstrap required, opened a new log
I20260806 12:32:57.260581 26323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39961 } }
I20260806 12:32:57.260810 26323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:57.260903 26323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1c0a4fef47406c9d9b94ec8a3a786d, State: Initialized, Role: FOLLOWER
I20260806 12:32:57.261368 26323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39961 } }
I20260806 12:32:57.261520 26323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:57.261611 26323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:57.261710 26323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:57.262557 26323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39961 } }
I20260806 12:32:57.262881 26323 leader_election.cc:302] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c1c0a4fef47406c9d9b94ec8a3a786d; no voters:
I20260806 12:32:57.263235 26323 leader_election.cc:288] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:32:57.263370 26328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:57.263619 26328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [term 1 LEADER]: Becoming Leader. State: Replica: 7c1c0a4fef47406c9d9b94ec8a3a786d, State: Running, Role: LEADER
I20260806 12:32:57.263967 26328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39961 } }
I20260806 12:32:57.264377 26323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:57.265625 26330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c1c0a4fef47406c9d9b94ec8a3a786d. Latest consensus state: current_term: 1 leader_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39961 } } }
I20260806 12:32:57.265686 26329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1c0a4fef47406c9d9b94ec8a3a786d" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 39961 } } }
I20260806 12:32:57.265755 26330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:57.265755 26329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:57.266186 26337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:57.269250 26337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:57.274829 26337 catalog_manager.cc:1347] Generated new cluster ID: 4ded6d865ca846ffb5b76790c3fee5de
I20260806 12:32:57.274904 26337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:57.290102 26337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:32:57.290989 26337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:57.294793 26337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d: Generated new TSK 0
I20260806 12:32:57.295449 26337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:32:57.298604 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:39961
--builtin_ntp_servers=127.6.152.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 12:32:57.417019 26347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:57.417259 26347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:57.417316 26347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:57.421470 26347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:57.421530 26347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:57.421584 26347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:57.426466 26347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:39961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:57.427510 26347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:57.428840 26347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:57.438155 26352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:57.438163 26355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:57.438378 26347 server_base.cc:1034] running on GCE node
W20260806 12:32:57.438297 26353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:57.438858 26347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:57.439518 26347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:57.440693 26347 hybrid_clock.cc:648] HybridClock initialized: now 1786019577440668 us; error 45 us; skew 500 ppm
I20260806 12:32:57.442360 26347 webserver.cc:460] Webserver started at http://127.6.152.129:34339/ using document root <none> and password file <none>
I20260806 12:32:57.442912 26347 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:57.443002 26347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:57.443272 26347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:57.445211 26347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "9e502467a4984ff28bdb23a0cc4af770"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "adbcf6f9bb199a0bb61899d682df83a4"
server_key_iv: "35cc2950fd3299b83bbff96abaa198ee"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.445761 26347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "9e502467a4984ff28bdb23a0cc4af770"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "adbcf6f9bb199a0bb61899d682df83a4"
server_key_iv: "35cc2950fd3299b83bbff96abaa198ee"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.449186 26347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:32:57.451401 26361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.452461 26347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:32:57.452594 26347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "9e502467a4984ff28bdb23a0cc4af770"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "adbcf6f9bb199a0bb61899d682df83a4"
server_key_iv: "35cc2950fd3299b83bbff96abaa198ee"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.452698 26347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:57.466004 26347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:57.466609 26347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:57.466804 26347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:57.467316 26347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:57.468170 26347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:57.468217 26347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.468263 26347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:57.468305 26347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.476790 26347 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:42843
I20260806 12:32:57.476852 26476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:42843 every 8 connection(s)
I20260806 12:32:57.477682 26347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:57.485450 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26347
I20260806 12:32:57.485586 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:32:57.485914 6754 external_mini_cluster.cc:1428] Setting key 8796dcd39133b0219c32b3fca8f5a98e
I20260806 12:32:57.486811 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.479449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:32773 (local address 127.6.152.190:39961)
0806 12:32:57.479585 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.479588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.480580 (+ 992us) server_negotiation.cc:407] Connection header received
0806 12:32:57.481968 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.481972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.482036 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.482133 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:57.483023 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.483615 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.484763 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.485006 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.485107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.486255 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:57.486258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:57.486260 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:57.486272 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:57.486307 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:57.486319 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:57.486400 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.486578 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.486639 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:32:57.487703 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.479812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:39961 (local address 127.6.152.129:32773)
0806 12:32:57.480438 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.480478 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.481754 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.482303 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.482310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.482389 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:57.482841 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.482854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.483746 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.483750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.484634 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.484644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.485158 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.485162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.485358 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.485975 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:32:57.485978 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:57.486031 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:32:57.486106 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:57.486422 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:57.486429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:57.486434 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.486565 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.486817 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":114,"threads_started":1}
I20260806 12:32:57.488000 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:39961
--builtin_ntp_servers=127.6.152.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 12:32:57.488996 26477 heartbeater.cc:347] Connected to a master server at 127.6.152.190:39961
I20260806 12:32:57.489215 26477 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:57.489679 26477 heartbeater.cc:511] Master 127.6.152.190:39961 requested a full tablet report, sending...
I20260806 12:32:57.491242 26286 ts_manager.cc:194] Registered new tserver with Master: 9e502467a4984ff28bdb23a0cc4af770 (127.6.152.129:42843)
I20260806 12:32:57.492918 26286 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.129:32773
W20260806 12:32:57.598250 26480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:57.598511 26480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:57.598551 26480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:57.602456 26480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:57.602510 26480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:57.602564 26480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:57.607412 26480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:39961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:57.608376 26480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:57.609632 26480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:57.618970 26488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:57.619031 26486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:57.619007 26485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:57.619323 26480 server_base.cc:1034] running on GCE node
I20260806 12:32:57.619727 26480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:57.620409 26480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:57.621592 26480 hybrid_clock.cc:648] HybridClock initialized: now 1786019577621578 us; error 25 us; skew 500 ppm
I20260806 12:32:57.623332 26480 webserver.cc:460] Webserver started at http://127.6.152.130:36571/ using document root <none> and password file <none>
I20260806 12:32:57.623891 26480 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:57.623984 26480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:57.624207 26480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:57.626196 26480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "6bdfc56a7a464c8fb65adfa08ef21a12"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "b4e143ee13b00a9ce81d9439a3134737"
server_key_iv: "190c0b38f1cc8183e9ff7fc85bb2a7be"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.626798 26480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "6bdfc56a7a464c8fb65adfa08ef21a12"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "b4e143ee13b00a9ce81d9439a3134737"
server_key_iv: "190c0b38f1cc8183e9ff7fc85bb2a7be"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.630210 26480 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 12:32:57.632477 26494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.633553 26480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 12:32:57.633675 26480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "6bdfc56a7a464c8fb65adfa08ef21a12"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "b4e143ee13b00a9ce81d9439a3134737"
server_key_iv: "190c0b38f1cc8183e9ff7fc85bb2a7be"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.633776 26480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:57.659334 26480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:57.660036 26480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:57.660235 26480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:57.660825 26480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:57.661783 26480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:57.661852 26480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.661924 26480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:57.661950 26480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.670801 26480 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:33127
I20260806 12:32:57.670840 26610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:33127 every 8 connection(s)
I20260806 12:32:57.671671 26480 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:32:57.675282 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26480
I20260806 12:32:57.675402 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:32:57.675673 6754 external_mini_cluster.cc:1428] Setting key 9ecb69c4399a20b6c237be1389396d1d
I20260806 12:32:57.677572 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:39961
--builtin_ntp_servers=127.6.152.148:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 12:32:57.680718 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.673380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:40259 (local address 127.6.152.190:39961)
0806 12:32:57.673520 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.673524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.674493 (+ 969us) server_negotiation.cc:407] Connection header received
0806 12:32:57.676009 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.676012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.676078 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.676174 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:57.677174 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.677705 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.678811 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.679037 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.679161 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.680236 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:57.680239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:57.680242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:57.680259 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:57.680278 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:57.680286 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:57.680358 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.680517 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.680581 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 12:32:57.681552 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.673701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:39961 (local address 127.6.152.130:40259)
0806 12:32:57.674340 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.674381 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.675801 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.676344 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.676353 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.676461 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:57.677031 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.677045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.677823 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.677827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.678683 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.678693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.679220 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.679225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.679379 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.679978 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:32:57.679980 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:57.680017 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:32:57.680076 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:57.680374 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:57.680388 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:57.680393 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.680515 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.680774 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":105,"threads_started":1}
I20260806 12:32:57.682603 26611 heartbeater.cc:347] Connected to a master server at 127.6.152.190:39961
I20260806 12:32:57.682828 26611 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:57.683369 26611 heartbeater.cc:511] Master 127.6.152.190:39961 requested a full tablet report, sending...
I20260806 12:32:57.684398 26286 ts_manager.cc:194] Registered new tserver with Master: 6bdfc56a7a464c8fb65adfa08ef21a12 (127.6.152.130:33127)
I20260806 12:32:57.685263 26286 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.130:40259
W20260806 12:32:57.788764 26614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:57.789013 26614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:57.789050 26614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:57.792922 26614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:57.792969 26614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:57.793018 26614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:32:57.797821 26614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:39961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:57.798775 26614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:57.800135 26614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:57.809212 26620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:57.809252 26622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:57.809363 26614 server_base.cc:1034] running on GCE node
W20260806 12:32:57.809574 26619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:57.809952 26614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:57.810554 26614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:57.811709 26614 hybrid_clock.cc:648] HybridClock initialized: now 1786019577811694 us; error 26 us; skew 500 ppm
I20260806 12:32:57.813522 26614 webserver.cc:460] Webserver started at http://127.6.152.131:44483/ using document root <none> and password file <none>
I20260806 12:32:57.814070 26614 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:57.814129 26614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:57.814366 26614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:57.816390 26614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "364b531ba5324615ab80d1fb4b18e4bb"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "199c307300207b33b8661082047d0a2d"
server_key_iv: "8d89d745cc0ae0b19732b7dea470b183"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.816948 26614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "364b531ba5324615ab80d1fb4b18e4bb"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "199c307300207b33b8661082047d0a2d"
server_key_iv: "8d89d745cc0ae0b19732b7dea470b183"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.820580 26614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:32:57.822871 26628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.824080 26614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:32:57.824235 26614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "364b531ba5324615ab80d1fb4b18e4bb"
format_stamp: "Formatted at 2026-08-06 12:32:57 on dist-test-slave-1xrh"
server_key: "199c307300207b33b8661082047d0a2d"
server_key_iv: "8d89d745cc0ae0b19732b7dea470b183"
server_key_version: "encryptionkey@0"
I20260806 12:32:57.824344 26614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:57.850062 26614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:57.850768 26614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:57.850965 26614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:57.851591 26614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:57.852545 26614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:57.852610 26614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.852679 26614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:57.852730 26614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:57.861670 26614 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:34445
I20260806 12:32:57.861706 26743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:34445 every 8 connection(s)
I20260806 12:32:57.862533 26614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:32:57.863943 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26614
I20260806 12:32:57.864050 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:32:57.864363 6754 external_mini_cluster.cc:1428] Setting key 33b61a592a0a5119924c3aa82e572007
I20260806 12:32:57.871748 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.864764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:40869 (local address 127.6.152.190:39961)
0806 12:32:57.864918 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.864923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.865669 (+ 746us) server_negotiation.cc:407] Connection header received
0806 12:32:57.867205 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.867208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.867266 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.867339 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:57.868167 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.868719 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.869875 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.870115 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.870225 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.871256 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:57.871260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:57.871263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:57.871276 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:57.871298 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:57.871311 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:57.871382 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.871551 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.871612 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 12:32:57.872575 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.864753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:39961 (local address 127.6.152.131:40869)
0806 12:32:57.865540 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.865579 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.866979 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.867502 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.867509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.867589 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:57.868035 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.868048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.868850 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.868853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.869749 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.869759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.870261 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.870265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.870427 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.870982 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:32:57.870983 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:57.871020 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:32:57.871079 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:57.871397 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:57.871406 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:57.871424 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.871549 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.871806 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":141,"threads_started":1}
I20260806 12:32:57.873600 26744 heartbeater.cc:347] Connected to a master server at 127.6.152.190:39961
I20260806 12:32:57.873826 26744 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:57.874330 26744 heartbeater.cc:511] Master 127.6.152.190:39961 requested a full tablet report, sending...
I20260806 12:32:57.875267 26286 ts_manager.cc:194] Registered new tserver with Master: 364b531ba5324615ab80d1fb4b18e4bb (127.6.152.131:34445)
I20260806 12:32:57.875828 26286 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.131:40869
I20260806 12:32:57.878970 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:32:57.884852 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.880874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.6.152.190:39961)
0806 12:32:57.881043 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.881047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.881210 (+ 163us) server_negotiation.cc:407] Connection header received
0806 12:32:57.881330 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.881333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.881380 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.881463 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:57.881748 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.882231 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.883107 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.883369 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.883474 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.884377 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:57.884382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:57.884384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:32:57.884401 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:57.884419 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:32:57.884428 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:57.884555 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.884685 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.884717 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 12:32:57.887549 26286 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:32:57.890450 26286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:32:57.904440 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.898382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42843 (local address 127.0.0.1:59984)
0806 12:32:57.899289 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.899337 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.899465 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.900550 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.900555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.900578 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:57.900695 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.900701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.901684 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.901690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.902843 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.902853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.903857 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.903860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.903981 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.904025 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.904150 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.904295 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":65,"threads_started":1}
I20260806 12:32:57.904474 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.898560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.6.152.129:42843)
0806 12:32:57.900143 (+ 1583us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.900157 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.900178 (+ 21us) server_negotiation.cc:407] Connection header received
0806 12:32:57.900244 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.900248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.900394 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.900565 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:57.900816 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.901570 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.902976 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.903703 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.903818 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.904013 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.904192 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.904296 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1475,"thread_start_us":108,"threads_started":1}
I20260806 12:32:57.904990 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.899015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:34445 (local address 127.0.0.1:46118)
0806 12:32:57.899936 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.899949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.900048 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.900643 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.900646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.900665 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:57.900777 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.900783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.902199 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.902202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.903357 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.903366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.904490 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.904496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.904619 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.904691 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.904763 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.904841 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":30,"thread_start_us":116,"threads_started":1}
I20260806 12:32:57.905019 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.898567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:33127 (local address 127.0.0.1:37156)
0806 12:32:57.899726 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.899739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.899857 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.900349 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.900362 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.900384 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:57.900522 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.900532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.901995 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.901998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.903238 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.903249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.904531 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.904535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.904665 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.904718 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.904805 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.904876 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":23,"thread_start_us":92,"threads_started":1}
I20260806 12:32:57.905386 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.899093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.6.152.130:33127)
0806 12:32:57.899556 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.899569 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.899878 (+ 309us) server_negotiation.cc:407] Connection header received
0806 12:32:57.899988 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.899995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.900204 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.900365 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:57.900667 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.901860 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.903369 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.904176 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.904279 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.904457 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.905115 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.905221 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":101,"threads_started":1}
I20260806 12:32:57.905882 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.899481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.6.152.131:34445)
0806 12:32:57.899865 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.899878 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.900048 (+ 170us) server_negotiation.cc:407] Connection header received
0806 12:32:57.900145 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.900152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.900332 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.900491 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:57.900891 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.902085 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.903468 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.904137 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.904232 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.904404 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.904814 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.904903 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":162,"threads_started":1}
I20260806 12:32:57.907949 26409 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac56e8024a5408db0638d60c43642e1 (DEFAULT_TABLE table=test-table [id=fc88f46db86f422cb6ce6e4808faf188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:57.908344 26676 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac56e8024a5408db0638d60c43642e1 (DEFAULT_TABLE table=test-table [id=fc88f46db86f422cb6ce6e4808faf188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:57.909112 26409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac56e8024a5408db0638d60c43642e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:57.909276 26676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac56e8024a5408db0638d60c43642e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:57.909116 26542 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac56e8024a5408db0638d60c43642e1 (DEFAULT_TABLE table=test-table [id=fc88f46db86f422cb6ce6e4808faf188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:32:57.909981 26542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac56e8024a5408db0638d60c43642e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:57.915746 26761 tablet_bootstrap.cc:492] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Bootstrap starting.
I20260806 12:32:57.917119 26762 tablet_bootstrap.cc:492] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Bootstrap starting.
I20260806 12:32:57.919360 26762 tablet_bootstrap.cc:654] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:57.919351 26761 tablet_bootstrap.cc:654] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:57.919430 26763 tablet_bootstrap.cc:492] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: Bootstrap starting.
I20260806 12:32:57.920255 26762 log.cc:824] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:57.920255 26761 log.cc:824] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:57.921275 26763 tablet_bootstrap.cc:654] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:57.921972 26763 log.cc:824] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:57.922380 26761 tablet_bootstrap.cc:492] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: No bootstrap required, opened a new log
I20260806 12:32:57.922380 26762 tablet_bootstrap.cc:492] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: No bootstrap required, opened a new log
I20260806 12:32:57.922571 26762 ts_tablet_manager.cc:1397] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 12:32:57.922585 26761 ts_tablet_manager.cc:1397] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 12:32:57.923890 26763 tablet_bootstrap.cc:492] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: No bootstrap required, opened a new log
I20260806 12:32:57.924083 26763 ts_tablet_manager.cc:1397] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 12:32:57.926993 26761 raft_consensus.cc:348] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.927034 26762 raft_consensus.cc:348] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.927290 26761 raft_consensus.cc:374] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:57.927297 26762 raft_consensus.cc:374] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:57.927388 26761 raft_consensus.cc:729] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bdfc56a7a464c8fb65adfa08ef21a12, State: Initialized, Role: FOLLOWER
I20260806 12:32:57.927418 26762 raft_consensus.cc:729] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 364b531ba5324615ab80d1fb4b18e4bb, State: Initialized, Role: FOLLOWER
I20260806 12:32:57.927403 26763 raft_consensus.cc:348] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.927659 26763 raft_consensus.cc:374] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:57.927747 26763 raft_consensus.cc:729] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e502467a4984ff28bdb23a0cc4af770, State: Initialized, Role: FOLLOWER
I20260806 12:32:57.927875 26761 consensus_queue.cc:260] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.927874 26762 consensus_queue.cc:260] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.928154 26763 consensus_queue.cc:260] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.928630 26611 heartbeater.cc:503] Master 127.6.152.190:39961 was elected leader, sending a full tablet report...
I20260806 12:32:57.928941 26761 ts_tablet_manager.cc:1428] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 12:32:57.928941 26762 ts_tablet_manager.cc:1428] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 12:32:57.929247 26744 heartbeater.cc:503] Master 127.6.152.190:39961 was elected leader, sending a full tablet report...
I20260806 12:32:57.929734 26477 heartbeater.cc:503] Master 127.6.152.190:39961 was elected leader, sending a full tablet report...
I20260806 12:32:57.930189 26763 ts_tablet_manager.cc:1428] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 12:32:57.978852 26478 tablet.cc:2369] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:32:57.991302 26767 raft_consensus.cc:482] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:32:57.991518 26767 raft_consensus.cc:504] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:57.992659 26767 leader_election.cc:288] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e502467a4984ff28bdb23a0cc4af770 (127.6.152.129:42843), 364b531ba5324615ab80d1fb4b18e4bb (127.6.152.131:34445)
I20260806 12:32:57.996775 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.993077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:50883 (local address 127.6.152.131:34445)
0806 12:32:57.993248 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.993254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.993449 (+ 195us) server_negotiation.cc:407] Connection header received
0806 12:32:57.993591 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.993594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.993638 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.993721 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:57.993997 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.994488 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.995574 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.996155 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.996261 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.996311 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.996561 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.996641 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 12:32:57.996775 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.992861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42843 (local address 127.6.152.130:45779)
0806 12:32:57.993220 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.993237 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.993346 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.993632 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.993635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.993651 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:57.993775 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.993781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.994514 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.994519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.995410 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.995421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.996381 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.996384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.996467 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.996485 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.996583 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.996650 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260806 12:32:57.996804 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.992968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:45779 (local address 127.6.152.129:42843)
0806 12:32:57.993086 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:57.993090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:57.993301 (+ 211us) server_negotiation.cc:407] Connection header received
0806 12:32:57.993487 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:57.993491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:57.993538 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:57.993620 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:32:57.993894 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.994401 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.995548 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.996139 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.996261 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.996311 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:57.996614 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 12:32:57.996695 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 12:32:57.996783 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:57.992961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:34445 (local address 127.6.152.130:50883)
0806 12:32:57.993358 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:57.993370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:57.993489 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:57.993734 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:57.993737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:57.993754 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:32:57.993888 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.993893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.994603 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.994607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.995451 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:57.995460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:57.996316 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:57.996322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:57.996420 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:57.996450 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 12:32:57.996516 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 12:32:57.996593 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":150,"threads_started":1}
I20260806 12:32:57.997400 26429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac56e8024a5408db0638d60c43642e1" candidate_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e502467a4984ff28bdb23a0cc4af770" is_pre_election: true
I20260806 12:32:57.997524 26697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac56e8024a5408db0638d60c43642e1" candidate_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "364b531ba5324615ab80d1fb4b18e4bb" is_pre_election: true
I20260806 12:32:57.997746 26429 raft_consensus.cc:2454] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bdfc56a7a464c8fb65adfa08ef21a12 in term 0.
I20260806 12:32:57.997834 26697 raft_consensus.cc:2454] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bdfc56a7a464c8fb65adfa08ef21a12 in term 0.
I20260806 12:32:57.998262 26495 leader_election.cc:302] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 364b531ba5324615ab80d1fb4b18e4bb, 6bdfc56a7a464c8fb65adfa08ef21a12, 9e502467a4984ff28bdb23a0cc4af770; no voters:
I20260806 12:32:57.998560 26767 raft_consensus.cc:2781] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:32:57.998652 26767 raft_consensus.cc:482] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:32:57.998713 26767 raft_consensus.cc:3037] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:57.999887 26767 raft_consensus.cc:504] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:58.000252 26767 leader_election.cc:288] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [CANDIDATE]: Term 1 election: Requested vote from peers 9e502467a4984ff28bdb23a0cc4af770 (127.6.152.129:42843), 364b531ba5324615ab80d1fb4b18e4bb (127.6.152.131:34445)
I20260806 12:32:58.000587 26429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac56e8024a5408db0638d60c43642e1" candidate_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e502467a4984ff28bdb23a0cc4af770"
I20260806 12:32:58.000592 26697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac56e8024a5408db0638d60c43642e1" candidate_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "364b531ba5324615ab80d1fb4b18e4bb"
I20260806 12:32:58.000715 26429 raft_consensus.cc:3037] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:58.000715 26697 raft_consensus.cc:3037] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:58.001726 26697 raft_consensus.cc:2454] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bdfc56a7a464c8fb65adfa08ef21a12 in term 1.
I20260806 12:32:58.001775 26429 raft_consensus.cc:2454] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bdfc56a7a464c8fb65adfa08ef21a12 in term 1.
I20260806 12:32:58.002065 26496 leader_election.cc:302] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 364b531ba5324615ab80d1fb4b18e4bb, 6bdfc56a7a464c8fb65adfa08ef21a12; no voters:
I20260806 12:32:58.002240 26767 raft_consensus.cc:2781] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:58.002449 26767 raft_consensus.cc:686] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 1 LEADER]: Becoming Leader. State: Replica: 6bdfc56a7a464c8fb65adfa08ef21a12, State: Running, Role: LEADER
I20260806 12:32:58.002790 26767 consensus_queue.cc:237] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } }
I20260806 12:32:58.006076 26285 catalog_manager.cc:5495] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bdfc56a7a464c8fb65adfa08ef21a12 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bdfc56a7a464c8fb65adfa08ef21a12" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 33127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 } health_report { overall_health: UNKNOWN } } }
I20260806 12:32:58.028039 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:58.024548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.6.152.130:33127)
0806 12:32:58.024667 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:58.024671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:58.024684 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:32:58.024766 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:58.024768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:58.024821 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:58.024909 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:32:58.025129 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:58.025691 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:58.026531 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:58.026778 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:58.026857 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:58.027168 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:32:58.027619 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:32:58.027730 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:58.027842 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 12:32:58.027875 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:32:58.033718 26697 raft_consensus.cc:1261] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 1 FOLLOWER]: Refusing update from remote peer 6bdfc56a7a464c8fb65adfa08ef21a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:58.033723 26429 raft_consensus.cc:1261] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770 [term 1 FOLLOWER]: Refusing update from remote peer 6bdfc56a7a464c8fb65adfa08ef21a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:32:58.034394 26771 consensus_queue.cc:1035] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e502467a4984ff28bdb23a0cc4af770" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 42843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:58.034580 26767 consensus_queue.cc:1035] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "364b531ba5324615ab80d1fb4b18e4bb" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 34445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:32:58.050555 26285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:58.050699 26285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:32:58.053031 26285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c1c0a4fef47406c9d9b94ec8a3a786d: Sending DeleteTablet for 3 replicas of tablet 0ac56e8024a5408db0638d60c43642e1
I20260806 12:32:58.053782 26676 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac56e8024a5408db0638d60c43642e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:58 UTC) from {username='slave'} at 127.0.0.1:46118
I20260806 12:32:58.053995 26542 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac56e8024a5408db0638d60c43642e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:58 UTC) from {username='slave'} at 127.0.0.1:37156
I20260806 12:32:58.054288 26409 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac56e8024a5408db0638d60c43642e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:32:58 UTC) from {username='slave'} at 127.0.0.1:59984
I20260806 12:32:58.054490 26785 tablet_replica.cc:330] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: stopping tablet replica
I20260806 12:32:58.054772 26786 tablet_replica.cc:330] T 0ac56e8024a5408db0638d60c43642e1 P 9e502467a4984ff28bdb23a0cc4af770: stopping tablet replica
I20260806 12:32:58.054772 26785 raft_consensus.cc:2229] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:32:58.054872 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26347
I20260806 12:32:58.055639 26785 raft_consensus.cc:2258] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:58.057250 26785 ts_tablet_manager.cc:1905] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:58.059966 26785 ts_tablet_manager.cc:1918] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:58.060047 26785 log.cc:1196] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/0ac56e8024a5408db0638d60c43642e1
I20260806 12:32:58.060341 26785 ts_tablet_manager.cc:1939] T 0ac56e8024a5408db0638d60c43642e1 P 6bdfc56a7a464c8fb65adfa08ef21a12: Deleting consensus metadata
I20260806 12:32:58.061290 26784 tablet_replica.cc:330] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: stopping tablet replica
I20260806 12:32:58.061378 26273 catalog_manager.cc:4841] TS 6bdfc56a7a464c8fb65adfa08ef21a12 (127.6.152.130:33127): tablet 0ac56e8024a5408db0638d60c43642e1 (table test-table [id=fc88f46db86f422cb6ce6e4808faf188]) successfully deleted
I20260806 12:32:58.061714 26784 raft_consensus.cc:2229] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:32:58.062036 26784 raft_consensus.cc:2258] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:32:58.063733 26784 ts_tablet_manager.cc:1905] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:32:58.066639 26784 ts_tablet_manager.cc:1918] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:32:58.066783 26784 log.cc:1196] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/0ac56e8024a5408db0638d60c43642e1
I20260806 12:32:58.067173 26784 ts_tablet_manager.cc:1939] T 0ac56e8024a5408db0638d60c43642e1 P 364b531ba5324615ab80d1fb4b18e4bb: Deleting consensus metadata
W20260806 12:32:58.067225 26270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:42843 (error 108)
I20260806 12:32:58.068228 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:58.067923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:42843 (local address 127.0.0.1:59994)
0806 12:32:58.068051 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:58.068127 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:42843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 12:32:58.068359 26271 catalog_manager.cc:4841] TS 364b531ba5324615ab80d1fb4b18e4bb (127.6.152.131:34445): tablet 0ac56e8024a5408db0638d60c43642e1 (table test-table [id=fc88f46db86f422cb6ce6e4808faf188]) successfully deleted
I20260806 12:32:58.068475 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26480
W20260806 12:32:58.068570 26270 catalog_manager.cc:4584] TS 9e502467a4984ff28bdb23a0cc4af770 (127.6.152.129:42843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ac56e8024a5408db0638d60c43642e1: Network error: Client connection negotiation failed: client connection to 127.6.152.129:42843: connect: Connection refused (error 111)
I20260806 12:32:58.075452 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26614
I20260806 12:32:58.082048 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26254
2026-08-06T12:32:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1033 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:32:58 dist-test-slave-1xrh krb5kdc[26790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:32:58 dist-test-slave-1xrh krb5kdc[26790](info): set up 2 sockets
Aug 06 12:32:58 dist-test-slave-1xrh krb5kdc[26790](info): commencing operation
krb5kdc: starting...
W20260806 12:32:59.219970 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:32:59 dist-test-slave-1xrh krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:32:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:32:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:32:59.340076 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37667
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:37667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:59.450716 26803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:59.450992 26803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:59.451061 26803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:59.454887 26803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:32:59.454967 26803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:59.455003 26803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:59.455036 26803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:32:59.455075 26803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:32:59.460083 26803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:37667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:59.461046 26803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:59.462018 26803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:59.470319 26808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:59.470279 26809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:59.470428 26803 server_base.cc:1034] running on GCE node
W20260806 12:32:59.470278 26811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:59.471102 26803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:59.472204 26803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:59.473362 26803 hybrid_clock.cc:648] HybridClock initialized: now 1786019579473344 us; error 31 us; skew 500 ppm
Aug 06 12:32:59 dist-test-slave-1xrh krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:59.476236 26803 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:32:59.477052 26803 webserver.cc:460] Webserver started at http://127.6.152.190:38145/ using document root <none> and password file <none>
I20260806 12:32:59.477582 26803 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:59.477643 26803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:59.477849 26803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:59.479871 26803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "1264b2cb593a4aa0858249bcfb99ff73"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "7b5c6b3afc14b0bb2a62ed0dee389142"
server_key_iv: "5952d70415fddd307847c52fadc9c189"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.480439 26803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "1264b2cb593a4aa0858249bcfb99ff73"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "7b5c6b3afc14b0bb2a62ed0dee389142"
server_key_iv: "5952d70415fddd307847c52fadc9c189"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.484017 26803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:32:59.486410 26819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:59.487607 26803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 12:32:59.487728 26803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "1264b2cb593a4aa0858249bcfb99ff73"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "7b5c6b3afc14b0bb2a62ed0dee389142"
server_key_iv: "5952d70415fddd307847c52fadc9c189"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.487834 26803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:59.507620 26803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:59.510700 26803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:59.510911 26803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:59.517884 26803 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:37667
I20260806 12:32:59.517912 26873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:37667 every 8 connection(s)
I20260806 12:32:59.518851 26803 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:32:59.521529 26874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:32:59.525506 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26803
I20260806 12:32:59.525646 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:32:59.525956 6754 external_mini_cluster.cc:1428] Setting key 51764110d63e9a910048c727c412bb68
I20260806 12:32:59.527364 26874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73: Bootstrap starting.
I20260806 12:32:59.529978 26874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73: Neither blocks nor log segments found. Creating new log.
I20260806 12:32:59.530692 26874 log.cc:824] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73: Log is configured to *not* fsync() on all Append() calls
I20260806 12:32:59.532662 26874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73: No bootstrap required, opened a new log
Aug 06 12:32:59 dist-test-slave-1xrh krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:59.537005 26874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264b2cb593a4aa0858249bcfb99ff73" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37667 } }
I20260806 12:32:59.537205 26874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:32:59.537279 26874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1264b2cb593a4aa0858249bcfb99ff73, State: Initialized, Role: FOLLOWER
I20260806 12:32:59.537741 26874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264b2cb593a4aa0858249bcfb99ff73" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37667 } }
I20260806 12:32:59.537881 26874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:32:59.537953 26874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:32:59.538069 26874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:32:59.538918 26874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264b2cb593a4aa0858249bcfb99ff73" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37667 } }
I20260806 12:32:59.539306 26874 leader_election.cc:302] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1264b2cb593a4aa0858249bcfb99ff73; no voters:
I20260806 12:32:59.539654 26874 leader_election.cc:288] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:32:59.539750 26879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:32:59.539969 26879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [term 1 LEADER]: Becoming Leader. State: Replica: 1264b2cb593a4aa0858249bcfb99ff73, State: Running, Role: LEADER
I20260806 12:32:59.540304 26879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264b2cb593a4aa0858249bcfb99ff73" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37667 } }
I20260806 12:32:59.540548 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:59.527366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.6.152.190:37667)
0806 12:32:59.527799 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:59.527818 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:59.527859 (+ 41us) server_negotiation.cc:407] Connection header received
0806 12:32:59.528717 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:59.528740 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:59.529143 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:59.529485 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:59.530187 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:59.531262 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:59.532165 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:59.532819 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:59.533010 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:59.535688 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:59.535693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:59.535705 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:59.535730 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:59.537482 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:59.537898 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:59.537904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:59.537910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:59.538051 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:59.538322 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:59.538325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:59.538327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:59.538714 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:59.538924 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:59.539182 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 12:32:59.539505 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":128,"threads_started":1}
I20260806 12:32:59.540908 26874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:32:59.541460 26881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1264b2cb593a4aa0858249bcfb99ff73. Latest consensus state: current_term: 1 leader_uuid: "1264b2cb593a4aa0858249bcfb99ff73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264b2cb593a4aa0858249bcfb99ff73" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37667 } } }
I20260806 12:32:59.541574 26881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:59.542215 26880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1264b2cb593a4aa0858249bcfb99ff73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1264b2cb593a4aa0858249bcfb99ff73" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 37667 } } }
I20260806 12:32:59.542299 26880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73 [sys.catalog]: This master's current role is: LEADER
I20260806 12:32:59.543442 26885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:32:59.546113 26885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:32:59.551759 26885 catalog_manager.cc:1347] Generated new cluster ID: 1b04230dcd3e409cabc9ba4eed35c4d4
I20260806 12:32:59.551818 26885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:32:59.564363 26885 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:32:59.565189 26885 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:32:59.572901 26885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73: Generated new TSK 0
I20260806 12:32:59.573619 26885 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.152.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
Entry for principal kudu/127.6.152.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab.
I20260806 12:32:59.609731 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37667
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:59.719575 26901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:59.719818 26901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:59.719861 26901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:59.723963 26901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:59.724016 26901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:59.724072 26901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:32:59.729238 26901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.129.keytab
--principal=kudu/127.6.152.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:59.730185 26901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:59.731179 26901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:59.740633 26906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:59.740614 26909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:59.740679 26907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:59.740814 26901 server_base.cc:1034] running on GCE node
I20260806 12:32:59.741314 26901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:59.741995 26901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:59.743178 26901 hybrid_clock.cc:648] HybridClock initialized: now 1786019579743148 us; error 43 us; skew 500 ppm
Aug 06 12:32:59 dist-test-slave-1xrh krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:59.745965 26901 init.cc:377] Logged in from keytab as kudu/127.6.152.129@KRBTEST.COM (short username kudu)
I20260806 12:32:59.746825 26901 webserver.cc:460] Webserver started at http://127.6.152.129:42953/ using document root <none> and password file <none>
I20260806 12:32:59.747426 26901 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:59.747491 26901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:59.747725 26901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:59.749678 26901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "272306f315844ebabb4071b08b9ad3fd"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "0c3cb905efb84b2c220a86eae73fa408"
server_key_iv: "5c224f122ad121fe20819c605e240e50"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.750241 26901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "272306f315844ebabb4071b08b9ad3fd"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "0c3cb905efb84b2c220a86eae73fa408"
server_key_iv: "5c224f122ad121fe20819c605e240e50"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.753862 26901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:32:59.756294 26916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:59.757475 26901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:32:59.757602 26901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "272306f315844ebabb4071b08b9ad3fd"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "0c3cb905efb84b2c220a86eae73fa408"
server_key_iv: "5c224f122ad121fe20819c605e240e50"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.757709 26901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:59.773413 26901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:59.776314 26901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:59.776520 26901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:59.777122 26901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:59.777930 26901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:59.778008 26901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:59.778079 26901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:59.778117 26901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:59.787385 26901 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:43963
I20260806 12:32:59.787412 27031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:43963 every 8 connection(s)
I20260806 12:32:59.788272 26901 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:32:59.795967 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 26901
I20260806 12:32:59.796135 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:32:59.796470 6754 external_mini_cluster.cc:1428] Setting key 2616932fc59261060820acc0cd158e22
Aug 06 12:32:59 dist-test-slave-1xrh krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.129@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:32:59.802402 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:59.789986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:42955 (local address 127.6.152.190:37667)
0806 12:32:59.790123 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:32:59.790127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:32:59.791036 (+ 909us) server_negotiation.cc:407] Connection header received
0806 12:32:59.792535 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:32:59.792538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:32:59.792600 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:32:59.792695 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:32:59.793873 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:59.794398 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:59.795605 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:32:59.795963 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:32:59.796088 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:59.798771 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:32:59.798778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:32:59.798784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:32:59.798819 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:32:59.800412 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:59.800894 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:59.800898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:59.800899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:59.800983 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:32:59.801337 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:32:59.801342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:32:59.801344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:32:59.801520 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:32:59.801619 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 12:32:59.802094 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 12:32:59.802221 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:32:59.803399 27034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:32:59.790357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37667 (local address 127.6.152.129:42955)
0806 12:32:59.790891 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 12:32:59.790934 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 12:32:59.792324 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:32:59.792874 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:32:59.792880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:32:59.793222 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:32:59.793726 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:59.793738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:59.794524 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:59.794527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:59.795482 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:32:59.795492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:32:59.796160 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:32:59.796164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:32:59.796349 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:32:59.796972 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:32:59.796988 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:32:59.798608 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:32:59.800526 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:59.800532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:59.800545 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:59.800794 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:59.801091 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:32:59.801094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:32:59.801096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:32:59.801246 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:32:59.801618 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:32:59.801625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:32:59.801855 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 12:32:59.802052 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 12:32:59.802373 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":172,"threads_started":1}
I20260806 12:32:59.804714 27032 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37667
I20260806 12:32:59.804960 27032 heartbeater.cc:464] Registering TS with master...
I20260806 12:32:59.805436 27032 heartbeater.cc:511] Master 127.6.152.190:37667 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.152.130@KRBTEST.COM; defaulting to no policy
I20260806 12:32:59.806908 26836 ts_manager.cc:194] Registered new tserver with Master: 272306f315844ebabb4071b08b9ad3fd (127.6.152.129:43963)
I20260806 12:32:59.808059 26836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.129@KRBTEST.COM'} at 127.6.152.129:42955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
Entry for principal kudu/127.6.152.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab.
I20260806 12:32:59.826090 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37667
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:32:59.934553 27038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:32:59.934796 27038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:32:59.934839 27038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:32:59.938782 27038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:32:59.938830 27038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:32:59.938920 27038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:32:59.944056 27038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.130.keytab
--principal=kudu/127.6.152.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:32:59.944958 27038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:32:59.945915 27038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:32:59.954838 27046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:59.954843 27043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:32:59.954922 27044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:32:59.955144 27038 server_base.cc:1034] running on GCE node
I20260806 12:32:59.955621 27038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:32:59.956185 27038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:32:59.957356 27038 hybrid_clock.cc:648] HybridClock initialized: now 1786019579957326 us; error 42 us; skew 500 ppm
Aug 06 12:32:59 dist-test-slave-1xrh krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:32:59.959945 27038 init.cc:377] Logged in from keytab as kudu/127.6.152.130@KRBTEST.COM (short username kudu)
I20260806 12:32:59.960806 27038 webserver.cc:460] Webserver started at http://127.6.152.130:41323/ using document root <none> and password file <none>
I20260806 12:32:59.961409 27038 fs_manager.cc:359] Metadata directory not provided
I20260806 12:32:59.961504 27038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:32:59.961706 27038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:32:59.963636 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "f9417ae55bde4828a824dc102beffafb"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "d77d950bb7d1cd5e6f4a1d81e6e33915"
server_key_iv: "37fb033a23c54a0957b9db603b5fb7a9"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.964188 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "f9417ae55bde4828a824dc102beffafb"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "d77d950bb7d1cd5e6f4a1d81e6e33915"
server_key_iv: "37fb033a23c54a0957b9db603b5fb7a9"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.967643 27038 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:32:59.971269 27053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:59.972460 27038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:32:59.972572 27038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "f9417ae55bde4828a824dc102beffafb"
format_stamp: "Formatted at 2026-08-06 12:32:59 on dist-test-slave-1xrh"
server_key: "d77d950bb7d1cd5e6f4a1d81e6e33915"
server_key_iv: "37fb033a23c54a0957b9db603b5fb7a9"
server_key_version: "encryptionkey@0"
I20260806 12:32:59.972677 27038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:32:59.987919 27038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:32:59.990617 27038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:32:59.990795 27038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:32:59.991505 27038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:32:59.992326 27038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:32:59.992370 27038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:32:59.992452 27038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:32:59.992486 27038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.001080 27038 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:43777
I20260806 12:33:00.001112 27166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:43777 every 8 connection(s)
I20260806 12:33:00.002009 27038 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
Aug 06 12:33:00 dist-test-slave-1xrh krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019579, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.130@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:00.011961 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27038
I20260806 12:33:00.012130 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:33:00.012533 6754 external_mini_cluster.cc:1428] Setting key fd57bf219dfbe774456037abccc9133f
I20260806 12:33:00.016593 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.003909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:58861 (local address 127.6.152.190:37667)
0806 12:33:00.004053 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.004057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.004928 (+ 871us) server_negotiation.cc:407] Connection header received
0806 12:33:00.006282 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.006286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.006345 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.006452 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:00.007666 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.008214 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.009360 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.009594 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.009702 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.012319 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:00.012324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:00.012328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:00.012359 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:00.014378 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:00.014874 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:00.014880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:00.014892 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:00.014988 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:00.015361 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:00.015367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:00.015372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:00.015634 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:00.015736 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.016294 (+ 558us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.016430 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 12:33:00.017450 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.004292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37667 (local address 127.6.152.130:58861)
0806 12:33:00.004760 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.004812 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.006066 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.006633 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.006640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.007010 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:00.007515 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.007528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.008348 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.008352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.009231 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.009240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.009748 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.009752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.009927 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.010541 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:00.010558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:00.012129 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:00.014495 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:00.014501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:00.014513 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:00.014775 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:00.015085 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:00.015087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:00.015088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:00.015270 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:00.015752 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:00.015758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:00.016002 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.016234 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.016593 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260806 12:33:00.018610 27167 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37667
I20260806 12:33:00.018859 27167 heartbeater.cc:464] Registering TS with master...
I20260806 12:33:00.019397 27167 heartbeater.cc:511] Master 127.6.152.190:37667 requested a full tablet report, sending...
I20260806 12:33:00.020463 26836 ts_manager.cc:194] Registered new tserver with Master: f9417ae55bde4828a824dc102beffafb (127.6.152.130:43777)
I20260806 12:33:00.021327 26836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.130@KRBTEST.COM'} at 127.6.152.130:58861
WARNING: no policy specified for kudu/127.6.152.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
Entry for principal kudu/127.6.152.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab.
I20260806 12:33:00.042593 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37667
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:00.154160 27173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:00.154383 27173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:00.154430 27173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:00.158339 27173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:00.158392 27173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:00.158447 27173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:33:00.163712 27173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.131.keytab
--principal=kudu/127.6.152.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:00.164590 27173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:00.165499 27173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:00.174731 27179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:00.174731 27181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:00.174762 27173 server_base.cc:1034] running on GCE node
W20260806 12:33:00.174718 27178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:00.175405 27173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:00.175993 27173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:00.177142 27173 hybrid_clock.cc:648] HybridClock initialized: now 1786019580177114 us; error 40 us; skew 500 ppm
Aug 06 12:33:00 dist-test-slave-1xrh krb5kdc[26790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019580, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:00.179836 27173 init.cc:377] Logged in from keytab as kudu/127.6.152.131@KRBTEST.COM (short username kudu)
I20260806 12:33:00.180645 27173 webserver.cc:460] Webserver started at http://127.6.152.131:33157/ using document root <none> and password file <none>
I20260806 12:33:00.181159 27173 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:00.181241 27173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:00.181493 27173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:00.183476 27173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "9a199fcf60ba44c9b8d0564443595535"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "6f7892dc512ed686a8df3a1b73280a1f"
server_key_iv: "8e553606273e9f0792cebbe8d4fb2f23"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.184031 27173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "9a199fcf60ba44c9b8d0564443595535"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "6f7892dc512ed686a8df3a1b73280a1f"
server_key_iv: "8e553606273e9f0792cebbe8d4fb2f23"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.187494 27173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:33:00.189715 27188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.190878 27173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:00.191016 27173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "9a199fcf60ba44c9b8d0564443595535"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "6f7892dc512ed686a8df3a1b73280a1f"
server_key_iv: "8e553606273e9f0792cebbe8d4fb2f23"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.191116 27173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:00.204751 27173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:00.207808 27173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:00.207996 27173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:00.208546 27173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:33:00.209327 27173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:33:00.209393 27173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.209470 27173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:33:00.209511 27173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.217775 27173 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:40009
I20260806 12:33:00.217796 27303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:40009 every 8 connection(s)
I20260806 12:33:00.218611 27173 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
Aug 06 12:33:00 dist-test-slave-1xrh krb5kdc[26790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019580, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.131@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:00.228935 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27173
I20260806 12:33:00.229086 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:33:00.229470 6754 external_mini_cluster.cc:1428] Setting key 4552b8f67b04fcac82f5103159022035
I20260806 12:33:00.231388 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.220350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:49625 (local address 127.6.152.190:37667)
0806 12:33:00.220491 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.220495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.221266 (+ 771us) server_negotiation.cc:407] Connection header received
0806 12:33:00.222454 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.222457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.222505 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.222578 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:00.223729 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.224277 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.225426 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.225638 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.225718 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.228022 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:00.228025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:00.228026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:00.228050 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:00.229483 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:00.230005 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:00.230008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:00.230009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:00.230081 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:00.230391 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:00.230396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:00.230397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:00.230566 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:00.230667 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.231097 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.231229 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:33:00.232108 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.220689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:37667 (local address 127.6.152.131:49625)
0806 12:33:00.221114 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.221154 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.222268 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.222728 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.222734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.223149 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:00.223602 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.223614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.224381 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.224383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.225316 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.225330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.225760 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.225763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.225890 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.226451 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:00.226467 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:00.227878 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:00.229613 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:00.229623 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:00.229633 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:00.229882 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:00.230169 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:00.230173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:00.230175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:00.230300 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:00.230666 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:00.230674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:00.230882 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.231055 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.231372 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":123,"threads_started":1}
I20260806 12:33:00.233085 27304 heartbeater.cc:347] Connected to a master server at 127.6.152.190:37667
I20260806 12:33:00.233309 27304 heartbeater.cc:464] Registering TS with master...
I20260806 12:33:00.233740 27304 heartbeater.cc:511] Master 127.6.152.190:37667 requested a full tablet report, sending...
I20260806 12:33:00.234655 26836 ts_manager.cc:194] Registered new tserver with Master: 9a199fcf60ba44c9b8d0564443595535 (127.6.152.131:40009)
I20260806 12:33:00.235265 26836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.152.131@KRBTEST.COM'} at 127.6.152.131:49625
I20260806 12:33:00.243177 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:33:00.251786 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.245077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.6.152.190:37667)
0806 12:33:00.245205 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.245209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.245355 (+ 146us) server_negotiation.cc:407] Connection header received
0806 12:33:00.245449 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.245451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.245486 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.245547 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:00.246181 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.246648 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.247617 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.247841 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.247915 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.249188 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:00.249191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:00.249193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:00.249213 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:00.250373 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:00.250722 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:00.250724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:00.250725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:00.250790 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:00.251012 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:00.251014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:00.251015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:00.251253 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:00.251351 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.251532 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.251612 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:33:00.254244 26836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:33:00.257157 26836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:33:00.271126 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.265078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:43777 (local address 127.0.0.1:43710)
0806 12:33:00.266188 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.266198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.266307 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.266961 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.266966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.266982 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:00.267069 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.267075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.268867 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.268872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.269651 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.269659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.270602 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.270711 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.270750 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.270845 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.270967 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"thread_start_us":72,"threads_started":1}
I20260806 12:33:00.271284 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.264948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:40009 (local address 127.0.0.1:58186)
0806 12:33:00.265951 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.265998 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.266109 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.267060 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.267062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.267075 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:00.267185 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.267191 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.268320 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.268325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.270002 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.270013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.270856 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.270968 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.270993 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.271063 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.271156 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":262,"thread_start_us":95,"threads_started":1}
I20260806 12:33:00.271751 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.265700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.6.152.130:43777)
0806 12:33:00.266065 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.266077 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.266291 (+ 214us) server_negotiation.cc:407] Connection header received
0806 12:33:00.266420 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.266427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.266628 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.266767 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:00.267200 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.268712 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.269781 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.270480 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270582 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.270711 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.271391 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.271605 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":103,"threads_started":1}
I20260806 12:33:00.271777 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.265091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.6.152.131:40009)
0806 12:33:00.266544 (+ 1453us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.266555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.266573 (+ 18us) server_negotiation.cc:407] Connection header received
0806 12:33:00.266633 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.266639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.266790 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.266922 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:00.267273 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.268190 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270137 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.270736 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270847 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.270971 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.271103 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.271646 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":142,"threads_started":1}
I20260806 12:33:00.272308 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.265700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.6.152.129:43963)
0806 12:33:00.266064 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.266079 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.266363 (+ 284us) server_negotiation.cc:407] Connection header received
0806 12:33:00.266480 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.266485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.266631 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.266773 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:00.267036 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.268263 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270126 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.270897 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.270991 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.271148 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.272060 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.272178 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":73,"threads_started":1}
I20260806 12:33:00.272579 27319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.265571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:43963 (local address 127.0.0.1:36386)
0806 12:33:00.266247 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.266259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.266368 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.266767 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.266772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.266792 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:00.266908 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.266915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.268386 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.268389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.270002 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.270013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.271204 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.271207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.271285 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.271315 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.271566 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.271647 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":144,"thread_start_us":101,"threads_started":1}
I20260806 12:33:00.274470 27101 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7779788ee4cd3bcea2350ffd88d1b (DEFAULT_TABLE table=test-table [id=6206c77241bc4dea83c0eb29fdb080dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:33:00.274912 27236 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7779788ee4cd3bcea2350ffd88d1b (DEFAULT_TABLE table=test-table [id=6206c77241bc4dea83c0eb29fdb080dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:33:00.274950 26966 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7779788ee4cd3bcea2350ffd88d1b (DEFAULT_TABLE table=test-table [id=6206c77241bc4dea83c0eb29fdb080dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:33:00.275651 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7779788ee4cd3bcea2350ffd88d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:00.275909 26966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7779788ee4cd3bcea2350ffd88d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:00.275928 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7779788ee4cd3bcea2350ffd88d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:00.281929 27322 tablet_bootstrap.cc:492] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Bootstrap starting.
I20260806 12:33:00.283460 27324 tablet_bootstrap.cc:492] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd: Bootstrap starting.
I20260806 12:33:00.284433 27322 tablet_bootstrap.cc:654] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:00.285107 27324 tablet_bootstrap.cc:654] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:00.285145 27322 log.cc:824] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:00.286006 27323 tablet_bootstrap.cc:492] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Bootstrap starting.
I20260806 12:33:00.286301 27324 log.cc:824] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:00.287243 27322 tablet_bootstrap.cc:492] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: No bootstrap required, opened a new log
I20260806 12:33:00.287428 27322 ts_tablet_manager.cc:1397] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 12:33:00.288015 27324 tablet_bootstrap.cc:492] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd: No bootstrap required, opened a new log
I20260806 12:33:00.288208 27324 ts_tablet_manager.cc:1397] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:33:00.290387 27322 raft_consensus.cc:348] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.290606 27322 raft_consensus.cc:374] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:00.290681 27322 raft_consensus.cc:729] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9417ae55bde4828a824dc102beffafb, State: Initialized, Role: FOLLOWER
I20260806 12:33:00.290959 27324 raft_consensus.cc:348] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.291216 27324 raft_consensus.cc:374] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:00.291152 27322 consensus_queue.cc:260] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.291316 27324 raft_consensus.cc:729] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272306f315844ebabb4071b08b9ad3fd, State: Initialized, Role: FOLLOWER
I20260806 12:33:00.291697 27324 consensus_queue.cc:260] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.291739 27167 heartbeater.cc:503] Master 127.6.152.190:37667 was elected leader, sending a full tablet report...
I20260806 12:33:00.292096 27322 ts_tablet_manager.cc:1428] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:33:00.292303 27032 heartbeater.cc:503] Master 127.6.152.190:37667 was elected leader, sending a full tablet report...
I20260806 12:33:00.292544 27324 ts_tablet_manager.cc:1428] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 12:33:00.294624 27323 tablet_bootstrap.cc:654] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:00.295902 27323 log.cc:824] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:00.298026 27323 tablet_bootstrap.cc:492] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: No bootstrap required, opened a new log
I20260806 12:33:00.298175 27323 ts_tablet_manager.cc:1397] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260806 12:33:00.300925 27323 raft_consensus.cc:348] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.301133 27323 raft_consensus.cc:374] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:00.301167 27323 raft_consensus.cc:729] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a199fcf60ba44c9b8d0564443595535, State: Initialized, Role: FOLLOWER
I20260806 12:33:00.301525 27323 consensus_queue.cc:260] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.302191 27304 heartbeater.cc:503] Master 127.6.152.190:37667 was elected leader, sending a full tablet report...
I20260806 12:33:00.302502 27323 ts_tablet_manager.cc:1428] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 12:33:00.329763 27330 raft_consensus.cc:482] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:33:00.329962 27330 raft_consensus.cc:504] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.331034 27330 leader_election.cc:288] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9417ae55bde4828a824dc102beffafb (127.6.152.130:43777), 272306f315844ebabb4071b08b9ad3fd (127.6.152.129:43963)
I20260806 12:33:00.335956 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.331494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:49175 (local address 127.6.152.130:43777)
0806 12:33:00.331625 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.331630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.331645 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:33:00.331703 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.331707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.331761 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.331827 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:00.332180 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.332982 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.334637 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.335276 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.335385 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.335439 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.335755 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.335827 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:33:00.335956 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.331389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:51275 (local address 127.6.152.129:43963)
0806 12:33:00.331524 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.331529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.331904 (+ 375us) server_negotiation.cc:407] Connection header received
0806 12:33:00.331982 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.331984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.332028 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.332097 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:00.332360 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.333073 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.334637 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.335260 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.335398 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.335440 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.335755 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.335827 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 12:33:00.336050 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.331288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:43777 (local address 127.6.152.131:49175)
0806 12:33:00.331416 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.331433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.331526 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.331864 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.331867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.331891 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:00.332035 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.332041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.333089 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.333091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.334510 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.334522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.335386 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.335390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.335582 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.335647 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.335727 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.335879 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":158,"spinlock_wait_cycles":4224}
I20260806 12:33:00.336158 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.331288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:43963 (local address 127.6.152.131:51275)
0806 12:33:00.331766 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.331777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.331890 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.332134 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.332137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.332159 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:00.332249 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.332252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.333169 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.333172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.334517 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.334522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.335386 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.335390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.335586 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.335648 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.335727 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.335879 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":316,"spinlock_wait_cycles":768,"thread_start_us":135,"threads_started":1}
I20260806 12:33:00.336735 26986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7779788ee4cd3bcea2350ffd88d1b" candidate_uuid: "9a199fcf60ba44c9b8d0564443595535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272306f315844ebabb4071b08b9ad3fd" is_pre_election: true
I20260806 12:33:00.336736 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7779788ee4cd3bcea2350ffd88d1b" candidate_uuid: "9a199fcf60ba44c9b8d0564443595535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9417ae55bde4828a824dc102beffafb" is_pre_election: true
I20260806 12:33:00.337033 27121 raft_consensus.cc:2454] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a199fcf60ba44c9b8d0564443595535 in term 0.
I20260806 12:33:00.337033 26986 raft_consensus.cc:2454] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a199fcf60ba44c9b8d0564443595535 in term 0.
I20260806 12:33:00.337538 27190 leader_election.cc:302] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 272306f315844ebabb4071b08b9ad3fd, 9a199fcf60ba44c9b8d0564443595535, f9417ae55bde4828a824dc102beffafb; no voters:
I20260806 12:33:00.337808 27330 raft_consensus.cc:2781] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:33:00.337903 27330 raft_consensus.cc:482] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:33:00.337980 27330 raft_consensus.cc:3037] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:00.338994 27330 raft_consensus.cc:504] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.339347 27330 leader_election.cc:288] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [CANDIDATE]: Term 1 election: Requested vote from peers f9417ae55bde4828a824dc102beffafb (127.6.152.130:43777), 272306f315844ebabb4071b08b9ad3fd (127.6.152.129:43963)
I20260806 12:33:00.339715 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7779788ee4cd3bcea2350ffd88d1b" candidate_uuid: "9a199fcf60ba44c9b8d0564443595535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9417ae55bde4828a824dc102beffafb"
I20260806 12:33:00.339740 26986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7779788ee4cd3bcea2350ffd88d1b" candidate_uuid: "9a199fcf60ba44c9b8d0564443595535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272306f315844ebabb4071b08b9ad3fd"
I20260806 12:33:00.339843 26986 raft_consensus.cc:3037] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:00.339839 27121 raft_consensus.cc:3037] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:00.341166 26986 raft_consensus.cc:2454] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a199fcf60ba44c9b8d0564443595535 in term 1.
I20260806 12:33:00.341166 27121 raft_consensus.cc:2454] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a199fcf60ba44c9b8d0564443595535 in term 1.
I20260806 12:33:00.341464 27190 leader_election.cc:302] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 272306f315844ebabb4071b08b9ad3fd, 9a199fcf60ba44c9b8d0564443595535, f9417ae55bde4828a824dc102beffafb; no voters:
I20260806 12:33:00.341635 27330 raft_consensus.cc:2781] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:00.341850 27330 raft_consensus.cc:686] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 1 LEADER]: Becoming Leader. State: Replica: 9a199fcf60ba44c9b8d0564443595535, State: Running, Role: LEADER
I20260806 12:33:00.342157 27330 consensus_queue.cc:237] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } }
I20260806 12:33:00.345320 26835 catalog_manager.cc:5495] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a199fcf60ba44c9b8d0564443595535 (127.6.152.131). New cstate: current_term: 1 leader_uuid: "9a199fcf60ba44c9b8d0564443595535" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a199fcf60ba44c9b8d0564443595535" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 40009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 } health_report { overall_health: UNKNOWN } } }
I20260806 12:33:00.367915 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.364127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58190 (local address 127.6.152.131:40009)
0806 12:33:00.364262 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.364266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.364307 (+ 41us) server_negotiation.cc:407] Connection header received
0806 12:33:00.364635 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.364637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.364688 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.364772 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:00.365108 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.365597 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.366430 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.366636 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.366707 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.367057 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:00.367499 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:00.367591 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.367696 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.367745 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:33:00.373826 26986 raft_consensus.cc:1261] T 69c7779788ee4cd3bcea2350ffd88d1b P 272306f315844ebabb4071b08b9ad3fd [term 1 FOLLOWER]: Refusing update from remote peer 9a199fcf60ba44c9b8d0564443595535: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:00.373826 27121 raft_consensus.cc:1261] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 1 FOLLOWER]: Refusing update from remote peer 9a199fcf60ba44c9b8d0564443595535: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:00.374601 27333 consensus_queue.cc:1035] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [LEADER]: Connected to new peer: Peer: permanent_uuid: "272306f315844ebabb4071b08b9ad3fd" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:00.374807 27330 consensus_queue.cc:1035] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9417ae55bde4828a824dc102beffafb" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:00.382062 27336 mvcc.cc:205] Tried to move back new op lower bound from 7315536201203339264 to 7315536201083895808. Current Snapshot: MvccSnapshot[applied={T|T < 7315536201203339264}]
I20260806 12:33:00.383229 27337 mvcc.cc:205] Tried to move back new op lower bound from 7315536201203339264 to 7315536201083895808. Current Snapshot: MvccSnapshot[applied={T|T < 7315536201203339264}]
I20260806 12:33:00.385076 27338 mvcc.cc:205] Tried to move back new op lower bound from 7315536201203339264 to 7315536201083895808. Current Snapshot: MvccSnapshot[applied={T|T < 7315536201203339264}]
I20260806 12:33:00.390225 26836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:33:00.390391 26836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:33:00.392685 26836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1264b2cb593a4aa0858249bcfb99ff73: Sending DeleteTablet for 3 replicas of tablet 69c7779788ee4cd3bcea2350ffd88d1b
I20260806 12:33:00.393383 27236 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7779788ee4cd3bcea2350ffd88d1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:33:00 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:58186
I20260806 12:33:00.393532 27101 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7779788ee4cd3bcea2350ffd88d1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:33:00 UTC) from {username='kudu', principal='kudu/127.6.152.190@KRBTEST.COM'} at 127.0.0.1:43710
I20260806 12:33:00.394105 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26901
I20260806 12:33:00.394261 27345 tablet_replica.cc:330] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: stopping tablet replica
I20260806 12:33:00.394593 27345 raft_consensus.cc:2229] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:33:00.394979 27345 raft_consensus.cc:2258] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:33:00.396755 27346 tablet_replica.cc:330] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: stopping tablet replica
I20260806 12:33:00.396847 27345 ts_tablet_manager.cc:1905] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:33:00.397082 27346 raft_consensus.cc:2229] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:33:00.397424 27346 raft_consensus.cc:2258] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:33:00.398926 27346 ts_tablet_manager.cc:1905] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:33:00.399755 27345 ts_tablet_manager.cc:1918] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:33:00.399902 27345 log.cc:1196] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/69c7779788ee4cd3bcea2350ffd88d1b
I20260806 12:33:00.400309 27345 ts_tablet_manager.cc:1939] T 69c7779788ee4cd3bcea2350ffd88d1b P 9a199fcf60ba44c9b8d0564443595535: Deleting consensus metadata
I20260806 12:33:00.401510 26820 catalog_manager.cc:4841] TS 9a199fcf60ba44c9b8d0564443595535 (127.6.152.131:40009): tablet 69c7779788ee4cd3bcea2350ffd88d1b (table test-table [id=6206c77241bc4dea83c0eb29fdb080dd]) successfully deleted
I20260806 12:33:00.402434 27346 ts_tablet_manager.cc:1918] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:33:00.402552 27346 log.cc:1196] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/69c7779788ee4cd3bcea2350ffd88d1b
I20260806 12:33:00.402920 27346 ts_tablet_manager.cc:1939] T 69c7779788ee4cd3bcea2350ffd88d1b P f9417ae55bde4828a824dc102beffafb: Deleting consensus metadata
I20260806 12:33:00.404112 26822 catalog_manager.cc:4841] TS f9417ae55bde4828a824dc102beffafb (127.6.152.130:43777): tablet 69c7779788ee4cd3bcea2350ffd88d1b (table test-table [id=6206c77241bc4dea83c0eb29fdb080dd]) successfully deleted
W20260806 12:33:00.405000 26821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:43963 (error 108)
I20260806 12:33:00.405237 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27038
I20260806 12:33:00.406481 27319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.406172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:43963 (local address 127.0.0.1:36396)
0806 12:33:00.406333 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.406397 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 12:33:00.406625 26821 catalog_manager.cc:4584] TS 272306f315844ebabb4071b08b9ad3fd (127.6.152.129:43963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69c7779788ee4cd3bcea2350ffd88d1b: Network error: Client connection negotiation failed: client connection to 127.6.152.129:43963: connect: Connection refused (error 111)
I20260806 12:33:00.412719 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27173
I20260806 12:33:00.419191 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 26803
2026-08-06T12:33:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2337 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T12:33:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:00Z Disabled control of system clock
I20260806 12:33:00.450361 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:43603
--webserver_interface=127.6.152.190
--webserver_port=0
--builtin_ntp_servers=127.6.152.148:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:43603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 12:33:00.560611 27351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:00.560874 27351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:00.560909 27351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:00.564930 27351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:00.564980 27351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:00.564996 27351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:00.565012 27351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:00.565056 27351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:00.569737 27351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:43603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:43603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:00.570739 27351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:00.572142 27351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:00.580488 27357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:00.580605 27359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:00.580513 27356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:00.580541 27351 server_base.cc:1034] running on GCE node
I20260806 12:33:00.581199 27351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:00.582217 27351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:00.583395 27351 hybrid_clock.cc:648] HybridClock initialized: now 1786019580583383 us; error 34 us; skew 500 ppm
I20260806 12:33:00.585119 27351 webserver.cc:460] Webserver started at http://127.6.152.190:44277/ using document root <none> and password file <none>
I20260806 12:33:00.585677 27351 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:00.585767 27351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:00.585987 27351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:00.588032 27351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "a6dc12920e88478c9cd96d144305b17f"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "df7e7902e6b76baaeabdf6fe04032a7e"
server_key_iv: "2713844219f40855abcdb89af74290ff"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.588583 27351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "a6dc12920e88478c9cd96d144305b17f"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "df7e7902e6b76baaeabdf6fe04032a7e"
server_key_iv: "2713844219f40855abcdb89af74290ff"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.592434 27351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 12:33:00.594769 27365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.596071 27351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:00.596200 27351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "a6dc12920e88478c9cd96d144305b17f"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "df7e7902e6b76baaeabdf6fe04032a7e"
server_key_iv: "2713844219f40855abcdb89af74290ff"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.596305 27351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:00.610561 27351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:00.611276 27351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:00.611459 27351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:00.618208 27351 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:43603
I20260806 12:33:00.618239 27417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:43603 every 8 connection(s)
I20260806 12:33:00.619110 27351 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:00.622150 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:00.625782 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27351
I20260806 12:33:00.626044 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:00.626363 6754 external_mini_cluster.cc:1428] Setting key f5545328cc9d4180c097dcd42e290054
I20260806 12:33:00.627970 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: Bootstrap starting.
I20260806 12:33:00.630074 27418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:00.630789 27418 log.cc:824] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:00.632499 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: No bootstrap required, opened a new log
I20260806 12:33:00.635320 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.627794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.6.152.190:43603)
0806 12:33:00.628432 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.628454 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.628488 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:33:00.629111 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.629127 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.629503 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.629817 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:00.630267 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.631308 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.632192 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.632684 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.632846 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.633699 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:00.633704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:00.633717 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:00.634072 (+ 355us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:00.634123 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:00.634142 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:00.634246 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.634389 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.634627 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":134,"threads_started":1}
I20260806 12:33:00.636191 27418 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dc12920e88478c9cd96d144305b17f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43603 } }
I20260806 12:33:00.636477 27418 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:00.636538 27418 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6dc12920e88478c9cd96d144305b17f, State: Initialized, Role: FOLLOWER
I20260806 12:33:00.636992 27418 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dc12920e88478c9cd96d144305b17f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43603 } }
I20260806 12:33:00.637143 27418 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:00.637207 27418 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:00.637298 27418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:00.638170 27418 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dc12920e88478c9cd96d144305b17f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43603 } }
I20260806 12:33:00.638499 27418 leader_election.cc:302] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6dc12920e88478c9cd96d144305b17f; no voters:
I20260806 12:33:00.638804 27418 leader_election.cc:288] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:00.638906 27423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:00.639199 27423 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [term 1 LEADER]: Becoming Leader. State: Replica: a6dc12920e88478c9cd96d144305b17f, State: Running, Role: LEADER
I20260806 12:33:00.639523 27423 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dc12920e88478c9cd96d144305b17f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43603 } }
I20260806 12:33:00.639940 27418 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:00.641176 27425 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6dc12920e88478c9cd96d144305b17f. Latest consensus state: current_term: 1 leader_uuid: "a6dc12920e88478c9cd96d144305b17f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dc12920e88478c9cd96d144305b17f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43603 } } }
I20260806 12:33:00.641175 27424 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6dc12920e88478c9cd96d144305b17f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dc12920e88478c9cd96d144305b17f" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43603 } } }
I20260806 12:33:00.641377 27424 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:00.641662 27425 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f [sys.catalog]: This master's current role is: LEADER
W20260806 12:33:00.644634 27439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:00.644722 27439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:00.644801 27434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:00.645597 27434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:00.650298 27434 catalog_manager.cc:1347] Generated new cluster ID: 5e4be0f57c214a2698e53bec59d02237
I20260806 12:33:00.650362 27434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:00.666759 27434 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:00.667688 27434 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:00.675094 27434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: Generated new TSK 0
I20260806 12:33:00.675719 27434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:00.680433 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.129:0
--local_ip_for_outbound_sockets=127.6.152.129
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43603
--builtin_ntp_servers=127.6.152.148:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 12:33:00.790980 27443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:00.791235 27443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:00.791298 27443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:00.795168 27443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:00.795208 27443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:00.795317 27443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.129
I20260806 12:33:00.800014 27443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.152.129
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.129
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:00.801085 27443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:00.802384 27443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:00.811919 27451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:00.811990 27443 server_base.cc:1034] running on GCE node
W20260806 12:33:00.811904 27449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:00.812114 27448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:00.812505 27443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:00.813100 27443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:00.814278 27443 hybrid_clock.cc:648] HybridClock initialized: now 1786019580814264 us; error 24 us; skew 500 ppm
I20260806 12:33:00.820422 27443 webserver.cc:460] Webserver started at http://127.6.152.129:43195/ using document root <none> and password file <none>
I20260806 12:33:00.820976 27443 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:00.821063 27443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:00.821282 27443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:00.823316 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data/instance:
uuid: "449c109369a84a3e818dd4398fb2247d"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "529b347c54c1f2b21aa6aa1f66f68215"
server_key_iv: "3337c731b34949282afc8e72674c4565"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.823881 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance:
uuid: "449c109369a84a3e818dd4398fb2247d"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "529b347c54c1f2b21aa6aa1f66f68215"
server_key_iv: "3337c731b34949282afc8e72674c4565"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.827464 27443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:33:00.829645 27458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.830750 27443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:00.830886 27443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal
uuid: "449c109369a84a3e818dd4398fb2247d"
format_stamp: "Formatted at 2026-08-06 12:33:00 on dist-test-slave-1xrh"
server_key: "529b347c54c1f2b21aa6aa1f66f68215"
server_key_iv: "3337c731b34949282afc8e72674c4565"
server_key_version: "encryptionkey@0"
I20260806 12:33:00.830999 27443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:00.850327 27443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:00.850901 27443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:00.851075 27443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:00.851606 27443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:33:00.852432 27443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:33:00.852496 27443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.852559 27443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:33:00.852605 27443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:00.860945 27443 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.129:45521
I20260806 12:33:00.860991 27572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.129:45521 every 8 connection(s)
I20260806 12:33:00.861753 27443 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/data/info.pb
I20260806 12:33:00.866986 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27443
I20260806 12:33:00.867137 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-0/wal/instance
I20260806 12:33:00.867431 6754 external_mini_cluster.cc:1428] Setting key 78b11e567eebd898308c80354cdca83f
I20260806 12:33:00.869314 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.130:0
--local_ip_for_outbound_sockets=127.6.152.130
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43603
--builtin_ntp_servers=127.6.152.148:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 12:33:00.870854 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.863378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.129:53341 (local address 127.6.152.190:43603)
0806 12:33:00.863526 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:00.863531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:00.864565 (+ 1034us) server_negotiation.cc:407] Connection header received
0806 12:33:00.865861 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:00.865863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:00.865909 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:00.866004 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:00.866822 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.867441 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.868915 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.869147 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.869259 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.870402 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:00.870410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:00.870413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:00.870423 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:00.870441 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:00.870450 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:00.870522 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:00.870664 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 12:33:00.870721 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:33:00.871701 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:00.863809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:43603 (local address 127.6.152.129:53341)
0806 12:33:00.864434 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:00.864472 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:00.865687 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:00.866170 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:00.866176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:00.866254 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:00.866691 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.866702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.867569 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.867576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.868759 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:00.868772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:00.869299 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:00.869303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:00.869521 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:00.870148 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:33:00.870150 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:00.870197 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:33:00.870251 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:00.870525 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:00.870537 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:00.870543 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:33:00.870663 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 12:33:00.870929 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260806 12:33:00.872789 27573 heartbeater.cc:347] Connected to a master server at 127.6.152.190:43603
I20260806 12:33:00.873003 27573 heartbeater.cc:464] Registering TS with master...
I20260806 12:33:00.873477 27573 heartbeater.cc:511] Master 127.6.152.190:43603 requested a full tablet report, sending...
I20260806 12:33:00.874853 27382 ts_manager.cc:194] Registered new tserver with Master: 449c109369a84a3e818dd4398fb2247d (127.6.152.129:45521)
I20260806 12:33:00.876026 27382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.129:53341
W20260806 12:33:00.983423 27576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:00.983660 27576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:00.983718 27576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:00.987804 27576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:00.987871 27576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:00.987972 27576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.130
I20260806 12:33:00.992689 27576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.152.130
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.130
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:00.993675 27576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:00.995060 27576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:01.005025 27585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:01.005048 27583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:01.005232 27576 server_base.cc:1034] running on GCE node
W20260806 12:33:01.005048 27582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:01.005697 27576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:01.006304 27576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:01.007454 27576 hybrid_clock.cc:648] HybridClock initialized: now 1786019581007424 us; error 43 us; skew 500 ppm
I20260806 12:33:01.009258 27576 webserver.cc:460] Webserver started at http://127.6.152.130:43171/ using document root <none> and password file <none>
I20260806 12:33:01.009791 27576 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:01.009850 27576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:01.010010 27576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:01.012037 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data/instance:
uuid: "6d65b76821124e7abd166514609b0b31"
format_stamp: "Formatted at 2026-08-06 12:33:01 on dist-test-slave-1xrh"
server_key: "08fc8f6dd47c670e532528e189d31be7"
server_key_iv: "1d11be5d167a9a7746b6669aa7c21046"
server_key_version: "encryptionkey@0"
I20260806 12:33:01.012559 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance:
uuid: "6d65b76821124e7abd166514609b0b31"
format_stamp: "Formatted at 2026-08-06 12:33:01 on dist-test-slave-1xrh"
server_key: "08fc8f6dd47c670e532528e189d31be7"
server_key_iv: "1d11be5d167a9a7746b6669aa7c21046"
server_key_version: "encryptionkey@0"
I20260806 12:33:01.016098 27576 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:33:01.018373 27592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:01.019556 27576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:01.019647 27576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal
uuid: "6d65b76821124e7abd166514609b0b31"
format_stamp: "Formatted at 2026-08-06 12:33:01 on dist-test-slave-1xrh"
server_key: "08fc8f6dd47c670e532528e189d31be7"
server_key_iv: "1d11be5d167a9a7746b6669aa7c21046"
server_key_version: "encryptionkey@0"
I20260806 12:33:01.019727 27576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:01.043418 27576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:01.044126 27576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:01.044318 27576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:01.044900 27576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:33:01.045861 27576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:33:01.045908 27576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:01.045976 27576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:33:01.046012 27576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:01.054455 27576 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.130:37815
I20260806 12:33:01.054515 27706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.130:37815 every 8 connection(s)
I20260806 12:33:01.055404 27576 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/data/info.pb
I20260806 12:33:01.056020 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27576
I20260806 12:33:01.056165 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal/instance
I20260806 12:33:01.056509 6754 external_mini_cluster.cc:1428] Setting key 22d6a547fe564d24790f02cba3f931cd
I20260806 12:33:01.058792 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.152.131:0
--local_ip_for_outbound_sockets=127.6.152.131
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43603
--builtin_ntp_servers=127.6.152.148:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 12:33:01.065701 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.057380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:60119 (local address 127.6.152.190:43603)
0806 12:33:01.057516 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.057521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.058790 (+ 1269us) server_negotiation.cc:407] Connection header received
0806 12:33:01.060896 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.060899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.060957 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.061039 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:01.061957 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.062503 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.063742 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.063989 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.064098 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.065195 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:01.065199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:01.065202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:01.065214 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:01.065240 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:01.065248 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:01.065321 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.065468 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.065534 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:33:01.066529 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.057773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:43603 (local address 127.6.152.130:60119)
0806 12:33:01.058616 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.058681 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.060670 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.061216 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.061222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.061304 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:01.061788 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.061799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.062629 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.062633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.063620 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.063630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.064154 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.064158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.064312 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.064932 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:33:01.064934 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:01.064973 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:33:01.065047 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:01.065324 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:01.065336 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:01.065343 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.065466 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.065756 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":155,"threads_started":1}
I20260806 12:33:01.067700 27707 heartbeater.cc:347] Connected to a master server at 127.6.152.190:43603
I20260806 12:33:01.067955 27707 heartbeater.cc:464] Registering TS with master...
I20260806 12:33:01.068471 27707 heartbeater.cc:511] Master 127.6.152.190:43603 requested a full tablet report, sending...
I20260806 12:33:01.069638 27382 ts_manager.cc:194] Registered new tserver with Master: 6d65b76821124e7abd166514609b0b31 (127.6.152.130:37815)
I20260806 12:33:01.070405 27382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.130:60119
W20260806 12:33:01.170143 27710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:01.170403 27710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:01.170442 27710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:01.174896 27710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:01.174953 27710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:01.175043 27710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.152.131
I20260806 12:33:01.180398 27710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.152.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.152.131
--webserver_port=0
--tserver_master_addrs=127.6.152.190:43603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.152.131
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:01.181453 27710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:01.183225 27710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:01.193246 27718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:01.193269 27716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:01.193251 27715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:01.193531 27710 server_base.cc:1034] running on GCE node
I20260806 12:33:01.193897 27710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:01.194526 27710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:01.195691 27710 hybrid_clock.cc:648] HybridClock initialized: now 1786019581195673 us; error 46 us; skew 500 ppm
I20260806 12:33:01.197477 27710 webserver.cc:460] Webserver started at http://127.6.152.131:37937/ using document root <none> and password file <none>
I20260806 12:33:01.198009 27710 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:01.198069 27710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:01.198285 27710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:01.200522 27710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data/instance:
uuid: "100a8835232e43bd980cb8f35afcac43"
format_stamp: "Formatted at 2026-08-06 12:33:01 on dist-test-slave-1xrh"
server_key: "65fd1694c66fea1eb45175403b8c608b"
server_key_iv: "89f8bc6e190d3308ad1b65515817fae8"
server_key_version: "encryptionkey@0"
I20260806 12:33:01.201112 27710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance:
uuid: "100a8835232e43bd980cb8f35afcac43"
format_stamp: "Formatted at 2026-08-06 12:33:01 on dist-test-slave-1xrh"
server_key: "65fd1694c66fea1eb45175403b8c608b"
server_key_iv: "89f8bc6e190d3308ad1b65515817fae8"
server_key_version: "encryptionkey@0"
I20260806 12:33:01.204973 27710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 12:33:01.207366 27725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:01.208639 27710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:01.208755 27710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal
uuid: "100a8835232e43bd980cb8f35afcac43"
format_stamp: "Formatted at 2026-08-06 12:33:01 on dist-test-slave-1xrh"
server_key: "65fd1694c66fea1eb45175403b8c608b"
server_key_iv: "89f8bc6e190d3308ad1b65515817fae8"
server_key_version: "encryptionkey@0"
I20260806 12:33:01.208891 27710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:01.229714 27710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:01.230325 27710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:01.230532 27710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:01.231072 27710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 12:33:01.231983 27710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 12:33:01.232029 27710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:01.232095 27710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 12:33:01.232136 27710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:01.241219 27710 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.131:36483
I20260806 12:33:01.241285 27840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.131:36483 every 8 connection(s)
I20260806 12:33:01.242125 27710 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/data/info.pb
I20260806 12:33:01.245520 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27710
I20260806 12:33:01.245659 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal/instance
I20260806 12:33:01.245990 6754 external_mini_cluster.cc:1428] Setting key 4fd73cbeec45c0349e7b5f6a11a64aa1
I20260806 12:33:01.251211 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.243960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.131:40571 (local address 127.6.152.190:43603)
0806 12:33:01.244094 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.244098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.245019 (+ 921us) server_negotiation.cc:407] Connection header received
0806 12:33:01.246778 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.246781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.246834 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.246923 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:01.247828 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.248339 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.249449 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.249674 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.249781 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.250743 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:01.250746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:01.250749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:01.250766 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:01.250784 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:01.250791 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:01.250862 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.251002 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.251058 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:33:01.251981 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.244279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:43603 (local address 127.6.152.131:40571)
0806 12:33:01.244895 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.244933 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.246579 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.247079 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.247085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.247219 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:01.247698 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.247710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.248470 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.248474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.249321 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.249330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.249818 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.249823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.249992 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.250516 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 12:33:01.250517 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:01.250552 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 12:33:01.250607 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:01.250864 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:01.250876 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:01.250882 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.251001 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.251290 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":124,"threads_started":1}
I20260806 12:33:01.252908 27841 heartbeater.cc:347] Connected to a master server at 127.6.152.190:43603
I20260806 12:33:01.253094 27841 heartbeater.cc:464] Registering TS with master...
I20260806 12:33:01.253517 27841 heartbeater.cc:511] Master 127.6.152.190:43603 requested a full tablet report, sending...
I20260806 12:33:01.254441 27382 ts_manager.cc:194] Registered new tserver with Master: 100a8835232e43bd980cb8f35afcac43 (127.6.152.131:36483)
I20260806 12:33:01.254923 27382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.152.131:40571
I20260806 12:33:01.259682 6754 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 12:33:01.264629 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.261430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.6.152.190:43603)
0806 12:33:01.261554 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.261558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.261729 (+ 171us) server_negotiation.cc:407] Connection header received
0806 12:33:01.261814 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.261816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.261848 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.261916 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:01.262110 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.262554 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.263398 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.263583 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.263654 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.264295 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:01.264298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:01.264299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:01.264309 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:01.264324 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:01.264331 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:01.264384 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.264451 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.264489 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:33:01.266983 27382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 12:33:01.269914 27382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 12:33:01.283018 27853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.277909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36483 (local address 127.0.0.1:34844)
0806 12:33:01.278899 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.278913 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.279017 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.279593 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.279596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.279609 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:01.279709 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.279714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.280630 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.280634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.281776 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.281785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.282515 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.282519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.282620 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.282656 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.282763 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.282877 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":76,"threads_started":1}
I20260806 12:33:01.283035 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.278237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34844 (local address 127.6.152.131:36483)
0806 12:33:01.278726 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.278737 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.279011 (+ 274us) server_negotiation.cc:407] Connection header received
0806 12:33:01.279149 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.279155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.279322 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.279469 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:01.279816 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.280519 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.281892 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.282391 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.282513 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.282688 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.282789 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.282877 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":170,"threads_started":1}
I20260806 12:33:01.283536 27855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.277620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.130:37815 (local address 127.0.0.1:33758)
0806 12:33:01.278665 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.278685 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.278764 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.279308 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.279313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.279335 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:01.279488 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.279497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.280975 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.280979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.281921 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.281931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.283113 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.283144 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.283247 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.283269 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.283337 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.283409 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":28,"thread_start_us":97,"threads_started":1}
I20260806 12:33:01.283659 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.278255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33758 (local address 127.6.152.130:37815)
0806 12:33:01.278711 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.278732 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.278763 (+ 31us) server_negotiation.cc:407] Connection header received
0806 12:33:01.278914 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.278919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.279150 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.279300 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:01.279622 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.280837 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.282053 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.282801 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.282898 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.283054 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.283377 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.283490 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":148,"threads_started":1}
I20260806 12:33:01.284847 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.277517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:45521 (local address 127.0.0.1:56716)
0806 12:33:01.278236 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.278281 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.278418 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.279473 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.279476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.279495 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:01.279589 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.279594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.280975 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.280979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.283075 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.283088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.284462 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.284468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.284561 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.284585 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.284656 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.284725 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":1003,"thread_start_us":75,"threads_started":1}
I20260806 12:33:01.284972 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.277632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.6.152.129:45521)
0806 12:33:01.278934 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.278948 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.278973 (+ 25us) server_negotiation.cc:407] Connection header received
0806 12:33:01.279052 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.279059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.279292 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.279499 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:01.279683 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.280837 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.283235 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.284111 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.284212 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.284398 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.284693 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.284787 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1195,"thread_start_us":99,"threads_started":1}
I20260806 12:33:01.286376 27641 tablet_service.cc:1467] Processing CreateTablet for tablet f1ced5a2dcbd467b8101e5f21d42ac56 (DEFAULT_TABLE table=test-table [id=dd6965b2f350414087ee190aea06835b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:33:01.286377 27773 tablet_service.cc:1467] Processing CreateTablet for tablet f1ced5a2dcbd467b8101e5f21d42ac56 (DEFAULT_TABLE table=test-table [id=dd6965b2f350414087ee190aea06835b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:33:01.287462 27773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ced5a2dcbd467b8101e5f21d42ac56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:01.287460 27641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ced5a2dcbd467b8101e5f21d42ac56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:01.287108 27507 tablet_service.cc:1467] Processing CreateTablet for tablet f1ced5a2dcbd467b8101e5f21d42ac56 (DEFAULT_TABLE table=test-table [id=dd6965b2f350414087ee190aea06835b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 12:33:01.288077 27507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ced5a2dcbd467b8101e5f21d42ac56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:01.293764 27858 tablet_bootstrap.cc:492] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: Bootstrap starting.
I20260806 12:33:01.295259 27861 tablet_bootstrap.cc:492] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Bootstrap starting.
I20260806 12:33:01.295259 27859 tablet_bootstrap.cc:492] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Bootstrap starting.
I20260806 12:33:01.295992 27858 tablet_bootstrap.cc:654] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:01.296710 27858 log.cc:824] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:01.297370 27859 tablet_bootstrap.cc:654] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:01.297370 27861 tablet_bootstrap.cc:654] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:01.298204 27859 log.cc:824] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:01.298204 27861 log.cc:824] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:01.298749 27858 tablet_bootstrap.cc:492] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: No bootstrap required, opened a new log
I20260806 12:33:01.298951 27858 ts_tablet_manager.cc:1397] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:33:01.300199 27861 tablet_bootstrap.cc:492] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: No bootstrap required, opened a new log
I20260806 12:33:01.300370 27861 ts_tablet_manager.cc:1397] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 12:33:01.300661 27859 tablet_bootstrap.cc:492] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: No bootstrap required, opened a new log
I20260806 12:33:01.300853 27859 ts_tablet_manager.cc:1397] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 12:33:01.302440 27858 raft_consensus.cc:348] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.302662 27858 raft_consensus.cc:374] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:01.302742 27858 raft_consensus.cc:729] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 449c109369a84a3e818dd4398fb2247d, State: Initialized, Role: FOLLOWER
I20260806 12:33:01.303205 27858 consensus_queue.cc:260] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.303884 27573 heartbeater.cc:503] Master 127.6.152.190:43603 was elected leader, sending a full tablet report...
I20260806 12:33:01.304154 27858 ts_tablet_manager.cc:1428] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:33:01.304510 27859 raft_consensus.cc:348] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.304807 27859 raft_consensus.cc:374] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:01.304906 27859 raft_consensus.cc:729] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d65b76821124e7abd166514609b0b31, State: Initialized, Role: FOLLOWER
I20260806 12:33:01.305156 27861 raft_consensus.cc:348] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.305400 27861 raft_consensus.cc:374] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:01.305460 27861 raft_consensus.cc:729] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 100a8835232e43bd980cb8f35afcac43, State: Initialized, Role: FOLLOWER
I20260806 12:33:01.305406 27859 consensus_queue.cc:260] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.305835 27861 consensus_queue.cc:260] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.306260 27859 ts_tablet_manager.cc:1428] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 12:33:01.306381 27841 heartbeater.cc:503] Master 127.6.152.190:43603 was elected leader, sending a full tablet report...
W20260806 12:33:01.306469 27708 tablet.cc:2369] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:33:01.306694 27861 ts_tablet_manager.cc:1428] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 12:33:01.307188 27707 heartbeater.cc:503] Master 127.6.152.190:43603 was elected leader, sending a full tablet report...
W20260806 12:33:01.362821 27574 tablet.cc:2369] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 12:33:01.493249 27842 tablet.cc:2369] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:33:01.593097 27866 raft_consensus.cc:482] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 12:33:01.593348 27866 raft_consensus.cc:504] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.594554 27866 leader_election.cc:288] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 449c109369a84a3e818dd4398fb2247d (127.6.152.129:45521), 100a8835232e43bd980cb8f35afcac43 (127.6.152.131:36483)
I20260806 12:33:01.598430 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.594936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:46179 (local address 127.6.152.131:36483)
0806 12:33:01.595200 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.595206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.595222 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:33:01.595388 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.595391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.595432 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.595516 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:01.595820 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.596322 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.597465 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.597859 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.597996 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.598050 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.598231 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.598294 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260806 12:33:01.598465 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.594811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.131:36483 (local address 127.6.152.130:46179)
0806 12:33:01.595096 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.595113 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.595250 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.595530 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.595533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.595549 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:01.595705 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.595710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.596437 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.596452 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.597337 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.597347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.597987 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.597991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.598110 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.598129 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.598212 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.598279 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":86,"thread_start_us":91,"threads_started":1}
I20260806 12:33:01.598472 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.594907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.152.130:57401 (local address 127.6.152.129:45521)
0806 12:33:01.595094 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.595100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.595282 (+ 182us) server_negotiation.cc:407] Connection header received
0806 12:33:01.595443 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.595445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.595488 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.595572 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 12:33:01.595863 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.596351 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.597468 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.597842 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.598003 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.598057 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.598280 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.598350 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 12:33:01.598495 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.594810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:45521 (local address 127.6.152.130:57401)
0806 12:33:01.595198 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.595209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:01.595325 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:01.595584 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:01.595586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:01.595602 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 12:33:01.595744 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.595750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.596462 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.596467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.597336 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:01.597345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.598004 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.598007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:01.598158 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.598176 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 12:33:01.598260 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 12:33:01.598330 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":69,"thread_start_us":113,"threads_started":1}
I20260806 12:33:01.599334 27794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ced5a2dcbd467b8101e5f21d42ac56" candidate_uuid: "6d65b76821124e7abd166514609b0b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "100a8835232e43bd980cb8f35afcac43" is_pre_election: true
I20260806 12:33:01.599459 27527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ced5a2dcbd467b8101e5f21d42ac56" candidate_uuid: "6d65b76821124e7abd166514609b0b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "449c109369a84a3e818dd4398fb2247d" is_pre_election: true
I20260806 12:33:01.599704 27794 raft_consensus.cc:2454] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d65b76821124e7abd166514609b0b31 in term 0.
I20260806 12:33:01.599776 27527 raft_consensus.cc:2454] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d65b76821124e7abd166514609b0b31 in term 0.
I20260806 12:33:01.600210 27595 leader_election.cc:302] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 100a8835232e43bd980cb8f35afcac43, 6d65b76821124e7abd166514609b0b31; no voters:
I20260806 12:33:01.600469 27866 raft_consensus.cc:2781] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 12:33:01.600590 27866 raft_consensus.cc:482] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 12:33:01.600651 27866 raft_consensus.cc:3037] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:01.601760 27866 raft_consensus.cc:504] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.602138 27866 leader_election.cc:288] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [CANDIDATE]: Term 1 election: Requested vote from peers 449c109369a84a3e818dd4398fb2247d (127.6.152.129:45521), 100a8835232e43bd980cb8f35afcac43 (127.6.152.131:36483)
I20260806 12:33:01.602394 27527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ced5a2dcbd467b8101e5f21d42ac56" candidate_uuid: "6d65b76821124e7abd166514609b0b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "449c109369a84a3e818dd4398fb2247d"
I20260806 12:33:01.602475 27794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ced5a2dcbd467b8101e5f21d42ac56" candidate_uuid: "6d65b76821124e7abd166514609b0b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "100a8835232e43bd980cb8f35afcac43"
I20260806 12:33:01.602525 27527 raft_consensus.cc:3037] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:01.602604 27794 raft_consensus.cc:3037] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:01.603670 27527 raft_consensus.cc:2454] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d65b76821124e7abd166514609b0b31 in term 1.
I20260806 12:33:01.603844 27794 raft_consensus.cc:2454] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d65b76821124e7abd166514609b0b31 in term 1.
I20260806 12:33:01.603998 27593 leader_election.cc:302] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 449c109369a84a3e818dd4398fb2247d, 6d65b76821124e7abd166514609b0b31; no voters:
I20260806 12:33:01.604182 27866 raft_consensus.cc:2781] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:01.604396 27866 raft_consensus.cc:686] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d65b76821124e7abd166514609b0b31, State: Running, Role: LEADER
I20260806 12:33:01.604703 27866 consensus_queue.cc:237] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } }
I20260806 12:33:01.607752 27382 catalog_manager.cc:5495] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d65b76821124e7abd166514609b0b31 (127.6.152.130). New cstate: current_term: 1 leader_uuid: "6d65b76821124e7abd166514609b0b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d65b76821124e7abd166514609b0b31" member_type: VOTER last_known_addr { host: "127.6.152.130" port: 37815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 } health_report { overall_health: UNKNOWN } } }
I20260806 12:33:01.643401 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.639673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33772 (local address 127.6.152.130:37815)
0806 12:33:01.639805 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:01.639810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:01.639825 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:33:01.639967 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:01.639970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:01.640029 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:01.640157 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:01.640494 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.640991 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.641975 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:01.642197 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:01.642303 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:01.642607 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:01.642993 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:01.643118 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:01.643207 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 12:33:01.643259 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:33:01.649541 27527 raft_consensus.cc:1261] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 1 FOLLOWER]: Refusing update from remote peer 6d65b76821124e7abd166514609b0b31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:01.649541 27794 raft_consensus.cc:1261] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 1 FOLLOWER]: Refusing update from remote peer 6d65b76821124e7abd166514609b0b31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:01.650393 27870 consensus_queue.cc:1035] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "449c109369a84a3e818dd4398fb2247d" member_type: VOTER last_known_addr { host: "127.6.152.129" port: 45521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:01.650620 27866 consensus_queue.cc:1035] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "100a8835232e43bd980cb8f35afcac43" member_type: VOTER last_known_addr { host: "127.6.152.131" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:01.657325 27875 mvcc.cc:205] Tried to move back new op lower bound from 7315536206428426240 to 7315536206255525888. Current Snapshot: MvccSnapshot[applied={T|T < 7315536206428426240}]
I20260806 12:33:01.665767 27380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:33:01.665910 27380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 12:33:01.668257 27380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6dc12920e88478c9cd96d144305b17f: Sending DeleteTablet for 3 replicas of tablet f1ced5a2dcbd467b8101e5f21d42ac56
I20260806 12:33:01.669020 27507 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ced5a2dcbd467b8101e5f21d42ac56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:33:01 UTC) from {username='slave'} at 127.0.0.1:56716
I20260806 12:33:01.669034 27641 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ced5a2dcbd467b8101e5f21d42ac56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:33:01 UTC) from {username='slave'} at 127.0.0.1:33758
I20260806 12:33:01.669497 27883 tablet_replica.cc:330] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: stopping tablet replica
I20260806 12:33:01.669523 27773 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ced5a2dcbd467b8101e5f21d42ac56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 12:33:01 UTC) from {username='slave'} at 127.0.0.1:34844
I20260806 12:33:01.669824 27883 raft_consensus.cc:2229] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 1 LEADER]: Raft consensus shutting down.
I20260806 12:33:01.670094 27885 tablet_replica.cc:330] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: stopping tablet replica
I20260806 12:33:01.670275 27883 raft_consensus.cc:2258] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:33:01.670461 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27443
I20260806 12:33:01.670475 27884 tablet_replica.cc:330] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d: stopping tablet replica
I20260806 12:33:01.670753 27884 raft_consensus.cc:2229] T f1ced5a2dcbd467b8101e5f21d42ac56 P 449c109369a84a3e818dd4398fb2247d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:33:01.671864 27883 ts_tablet_manager.cc:1905] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:33:01.674703 27883 ts_tablet_manager.cc:1918] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:33:01.674790 27883 log.cc:1196] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-1/wal/wals/f1ced5a2dcbd467b8101e5f21d42ac56
I20260806 12:33:01.675103 27883 ts_tablet_manager.cc:1939] T f1ced5a2dcbd467b8101e5f21d42ac56 P 6d65b76821124e7abd166514609b0b31: Deleting consensus metadata
I20260806 12:33:01.676235 27367 catalog_manager.cc:4841] TS 6d65b76821124e7abd166514609b0b31 (127.6.152.130:37815): tablet f1ced5a2dcbd467b8101e5f21d42ac56 (table test-table [id=dd6965b2f350414087ee190aea06835b]) successfully deleted
I20260806 12:33:01.676275 27885 raft_consensus.cc:2229] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 12:33:01.676791 27885 raft_consensus.cc:2258] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 12:33:01.678521 27885 ts_tablet_manager.cc:1905] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 12:33:01.681085 27885 ts_tablet_manager.cc:1918] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 12:33:01.681180 27885 log.cc:1196] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Deleting WAL directory at /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786019324359312-6754-0/minicluster-data/ts-2/wal/wals/f1ced5a2dcbd467b8101e5f21d42ac56
I20260806 12:33:01.681560 27885 ts_tablet_manager.cc:1939] T f1ced5a2dcbd467b8101e5f21d42ac56 P 100a8835232e43bd980cb8f35afcac43: Deleting consensus metadata
I20260806 12:33:01.682724 27368 catalog_manager.cc:4841] TS 100a8835232e43bd980cb8f35afcac43 (127.6.152.131:36483): tablet f1ced5a2dcbd467b8101e5f21d42ac56 (table test-table [id=dd6965b2f350414087ee190aea06835b]) successfully deleted
W20260806 12:33:01.683112 27366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.152.129:45521 (error 108)
I20260806 12:33:01.684012 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27576
I20260806 12:33:01.684202 27853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:01.683889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.129:45521 (local address 127.0.0.1:56728)
0806 12:33:01.684016 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:01.684101 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.129:45521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 12:33:01.685207 27366 catalog_manager.cc:4584] TS 449c109369a84a3e818dd4398fb2247d (127.6.152.129:45521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1ced5a2dcbd467b8101e5f21d42ac56: Network error: Client connection negotiation failed: client connection to 127.6.152.129:45521: connect: Connection refused (error 111)
I20260806 12:33:01.690376 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27710
I20260806 12:33:01.696270 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27351
2026-08-06T12:33:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1275 ms)
[----------] 6 tests from EncryptionPolicyTest (11150 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:01 dist-test-slave-1xrh krb5kdc[27889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:01 dist-test-slave-1xrh krb5kdc[27889](info): set up 2 sockets
Aug 06 12:33:01 dist-test-slave-1xrh krb5kdc[27889](info): commencing operation
krb5kdc: starting...
W20260806 12:33:02.801662 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:02 dist-test-slave-1xrh krb5kdc[27889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:02.924386 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:03.035961 27902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:03.036252 27902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:03.036331 27902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:03.040357 27902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:03.040443 27902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:03.040480 27902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:03.040518 27902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:03.040553 27902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:03.045969 27902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33665
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:03.046983 27902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:03.048003 27902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:03.056676 27910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:03.056689 27907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:03.057030 27902 server_base.cc:1034] running on GCE node
W20260806 12:33:03.056825 27908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:03.057648 27902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:03.058667 27902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:03.059844 27902 hybrid_clock.cc:648] HybridClock initialized: now 1786019583059826 us; error 31 us; skew 500 ppm
Aug 06 12:33:03 dist-test-slave-1xrh krb5kdc[27889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019583, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:03.062556 27902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:03.063491 27902 webserver.cc:460] Webserver started at http://127.0.0.1:43875/ using document root <none> and password file <none>
I20260806 12:33:03.064035 27902 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:03.064123 27902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:03.064343 27902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:03.066373 27902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b"
format_stamp: "Formatted at 2026-08-06 12:33:03 on dist-test-slave-1xrh"
server_key: "f20fc82f5826c9c21ea39156dffbb260"
server_key_iv: "c342f82d2a54cbd8224919e27a7e08c7"
server_key_version: "encryptionkey@0"
I20260806 12:33:03.066962 27902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b"
format_stamp: "Formatted at 2026-08-06 12:33:03 on dist-test-slave-1xrh"
server_key: "f20fc82f5826c9c21ea39156dffbb260"
server_key_iv: "c342f82d2a54cbd8224919e27a7e08c7"
server_key_version: "encryptionkey@0"
I20260806 12:33:03.070642 27902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:33:03.073167 27917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:03.074301 27902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:03.074446 27902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b"
format_stamp: "Formatted at 2026-08-06 12:33:03 on dist-test-slave-1xrh"
server_key: "f20fc82f5826c9c21ea39156dffbb260"
server_key_iv: "c342f82d2a54cbd8224919e27a7e08c7"
server_key_version: "encryptionkey@0"
I20260806 12:33:03.074560 27902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:03.095456 27902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:03.098783 27902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:03.098999 27902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:03.106829 27902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33665
I20260806 12:33:03.106858 27969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33665 every 8 connection(s)
I20260806 12:33:03.107805 27902 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:03.110237 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 27902
I20260806 12:33:03.110396 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:03.110656 27970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:03.110699 6754 external_mini_cluster.cc:1428] Setting key d825e205720ce3e83489bb7cf5d1984a
I20260806 12:33:03.118091 27970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b: Bootstrap starting.
Aug 06 12:33:03 dist-test-slave-1xrh krb5kdc[27889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 12:33:03.120597 27970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:03.121515 27970 log.cc:824] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:03.123754 27970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b: No bootstrap required, opened a new log
I20260806 12:33:03.126236 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:03.112158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50477 (local address 127.0.0.1:33665)
0806 12:33:03.112591 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:03.112610 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:03.112653 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:33:03.113486 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:03.113511 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:03.113951 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:03.114348 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:03.115007 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:03.116516 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:03.117367 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:03.118052 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:03.118220 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:03.121011 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:03.121017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:03.121031 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:03.121061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:03.123033 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:03.123556 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:03.123562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:03.123568 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:03.123703 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:03.124060 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:03.124063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:03.124065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:03.124567 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:03.124787 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:03.125005 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 12:33:03.125229 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":134,"threads_started":1}
I20260806 12:33:03.128211 27970 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260806 12:33:03.128484 27970 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:03.128552 27970 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d99bf8f18d4eac9e035fcc73f8d55b, State: Initialized, Role: FOLLOWER
I20260806 12:33:03.129002 27970 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260806 12:33:03.129148 27970 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:03.129218 27970 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:03.129251 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:03.129312 27970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:03.130436 27970 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260806 12:33:03.130826 27970 leader_election.cc:302] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4d99bf8f18d4eac9e035fcc73f8d55b; no voters:
I20260806 12:33:03.131191 27970 leader_election.cc:288] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:03.131297 27980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:03.131533 27980 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [term 1 LEADER]: Becoming Leader. State: Replica: d4d99bf8f18d4eac9e035fcc73f8d55b, State: Running, Role: LEADER
I20260806 12:33:03.131899 27980 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260806 12:33:03.133000 27970 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:03.133736 27982 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4d99bf8f18d4eac9e035fcc73f8d55b. Latest consensus state: current_term: 1 leader_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } } }
I20260806 12:33:03.133859 27982 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:03.133795 27981 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4d99bf8f18d4eac9e035fcc73f8d55b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } } }
I20260806 12:33:03.133925 27981 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b [sys.catalog]: This master's current role is: LEADER
W20260806 12:33:03.138171 27996 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:03.138307 27996 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:03.138391 27987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:03.139097 27987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:03.140658 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:03.131997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52365 (local address 127.0.0.1:33665)
0806 12:33:03.132104 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:03.132109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:03.132124 (+ 15us) server_negotiation.cc:407] Connection header received
0806 12:33:03.132208 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:03.132211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:03.132252 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:03.132336 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:03.133243 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:03.134088 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:03.135245 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:03.135591 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:03.135694 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:03.137262 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:03.137266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:03.137269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:03.137294 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:03.138831 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:03.139372 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:03.139375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:03.139377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:03.139461 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:03.139789 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:03.139792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:03.139794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:03.139968 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:03.140115 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:03.140380 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 12:33:03.140481 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 12:33:03.145200 27987 catalog_manager.cc:1347] Generated new cluster ID: dbf254f63b30440984b6d6cf360ee275
I20260806 12:33:03.145267 27987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:03.156416 27987 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:03.157353 27987 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:03.167629 27987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4d99bf8f18d4eac9e035fcc73f8d55b: Generated new TSK 0
I20260806 12:33:03.168293 27987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:03.174023 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 27902
W20260806 12:33:03.181022 27887 connection.cc:535] client connection to 127.0.0.1:33665 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33665): Connection reset by peer (error 104)
2026-08-06T12:33:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:03 dist-test-slave-1xrh krb5kdc[28003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:03 dist-test-slave-1xrh krb5kdc[28003](info): set up 2 sockets
Aug 06 12:33:03 dist-test-slave-1xrh krb5kdc[28003](info): commencing operation
krb5kdc: starting...
W20260806 12:33:04.255512 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:04 dist-test-slave-1xrh krb5kdc[28003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:04.373768 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:04.482937 28016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:04.483235 28016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:04.483325 28016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:04.487259 28016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:04.487344 28016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:04.487376 28016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:04.487404 28016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:04.487443 28016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:04.492512 28016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46309
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:04.493502 28016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:04.494441 28016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:04.502755 28024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:04.502789 28016 server_base.cc:1034] running on GCE node
W20260806 12:33:04.503016 28022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:04.502837 28021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:04.503676 28016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:04.504714 28016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:04.505903 28016 hybrid_clock.cc:648] HybridClock initialized: now 1786019584505889 us; error 25 us; skew 500 ppm
Aug 06 12:33:04 dist-test-slave-1xrh krb5kdc[28003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:04.508558 28016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:04.509447 28016 webserver.cc:460] Webserver started at http://127.0.0.1:40433/ using document root <none> and password file <none>
I20260806 12:33:04.509985 28016 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:04.510070 28016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:04.510298 28016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:04.512269 28016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "4fb0bc7afa584bcc881394ec099bc247"
format_stamp: "Formatted at 2026-08-06 12:33:04 on dist-test-slave-1xrh"
server_key: "af7094ad11ceae34069a4a9e13a97e90"
server_key_iv: "383b7c62bd41391bef12885eed3b2d1e"
server_key_version: "encryptionkey@0"
I20260806 12:33:04.512811 28016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "4fb0bc7afa584bcc881394ec099bc247"
format_stamp: "Formatted at 2026-08-06 12:33:04 on dist-test-slave-1xrh"
server_key: "af7094ad11ceae34069a4a9e13a97e90"
server_key_iv: "383b7c62bd41391bef12885eed3b2d1e"
server_key_version: "encryptionkey@0"
I20260806 12:33:04.516553 28016 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:33:04.519019 28031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:04.520282 28016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:04.520429 28016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "4fb0bc7afa584bcc881394ec099bc247"
format_stamp: "Formatted at 2026-08-06 12:33:04 on dist-test-slave-1xrh"
server_key: "af7094ad11ceae34069a4a9e13a97e90"
server_key_iv: "383b7c62bd41391bef12885eed3b2d1e"
server_key_version: "encryptionkey@0"
I20260806 12:33:04.520545 28016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:04.544766 28016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:04.547802 28016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:04.548012 28016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:04.555620 28016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46309
I20260806 12:33:04.555650 28084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46309 every 8 connection(s)
I20260806 12:33:04.556569 28016 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:04.559316 28085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:04.559333 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28016
I20260806 12:33:04.559518 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:04.559844 6754 external_mini_cluster.cc:1428] Setting key 855abe873be4841e2cb060b4398354ba
I20260806 12:33:04.565904 28085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247: Bootstrap starting.
I20260806 12:33:04.568161 28085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:04.568783 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:04.561253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56975 (local address 127.0.0.1:46309)
0806 12:33:04.561655 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:04.561678 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:04.561712 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:33:04.562394 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:04.562412 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:04.562745 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:04.563010 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:04.563367 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:04.564341 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:04.565526 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:04.565962 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:04.566135 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:04.567076 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:04.567081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:04.567093 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:04.567534 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:04.567586 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:04.567609 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:04.567745 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:04.567928 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 12:33:04.568068 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":148,"threads_started":1}
I20260806 12:33:04.568943 28085 log.cc:824] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:04.570938 28085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247: No bootstrap required, opened a new log
I20260806 12:33:04.571683 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:04.574923 28085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb0bc7afa584bcc881394ec099bc247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46309 } }
I20260806 12:33:04.575222 28085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:04.575291 28085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb0bc7afa584bcc881394ec099bc247, State: Initialized, Role: FOLLOWER
I20260806 12:33:04.575747 28085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb0bc7afa584bcc881394ec099bc247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46309 } }
I20260806 12:33:04.575894 28085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:04.575958 28085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:04.576053 28085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:04.577162 28085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb0bc7afa584bcc881394ec099bc247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46309 } }
I20260806 12:33:04.577534 28085 leader_election.cc:302] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fb0bc7afa584bcc881394ec099bc247; no voters:
I20260806 12:33:04.577876 28085 leader_election.cc:288] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:04.577896 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:04.573651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45923 (local address 127.0.0.1:46309)
0806 12:33:04.573770 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:04.573776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:04.573971 (+ 195us) server_negotiation.cc:407] Connection header received
0806 12:33:04.574082 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:04.574084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:04.574119 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:04.574180 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:04.574470 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:04.574970 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:04.576159 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:04.576507 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:04.576588 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:04.577416 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:04.577419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:04.577421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:04.577432 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:04.577464 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:04.577474 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:04.577542 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:04.577714 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 12:33:04.577750 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:33:04.577963 28095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:04.578166 28095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [term 1 LEADER]: Becoming Leader. State: Replica: 4fb0bc7afa584bcc881394ec099bc247, State: Running, Role: LEADER
I20260806 12:33:04.578502 28095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb0bc7afa584bcc881394ec099bc247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46309 } }
I20260806 12:33:04.579187 28085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:04.579695 28097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fb0bc7afa584bcc881394ec099bc247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb0bc7afa584bcc881394ec099bc247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46309 } } }
I20260806 12:33:04.579742 28096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fb0bc7afa584bcc881394ec099bc247. Latest consensus state: current_term: 1 leader_uuid: "4fb0bc7afa584bcc881394ec099bc247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fb0bc7afa584bcc881394ec099bc247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46309 } } }
I20260806 12:33:04.579898 28096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:04.579888 28097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:04.580298 28102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:04.583217 28102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:04.588164 28102 catalog_manager.cc:1347] Generated new cluster ID: 2d8183d0678249f592bc53c5445c536e
I20260806 12:33:04.588227 28102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:04.598522 28102 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:04.599781 28102 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:04.612504 28102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fb0bc7afa584bcc881394ec099bc247: Generated new TSK 0
I20260806 12:33:04.613312 28102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:04.624569 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28016
W20260806 12:33:04.631115 28000 connection.cc:535] client connection to 127.0.0.1:46309 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46309): Connection reset by peer (error 104)
2026-08-06T12:33:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:04 dist-test-slave-1xrh krb5kdc[28116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:04 dist-test-slave-1xrh krb5kdc[28116](info): set up 2 sockets
Aug 06 12:33:04 dist-test-slave-1xrh krb5kdc[28116](info): commencing operation
krb5kdc: starting...
W20260806 12:33:05.671470 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:05 dist-test-slave-1xrh krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:05.794076 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:05.906152 28129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:05.906503 28129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:05.906622 28129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:05.910982 28129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:05.911067 28129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:05.911105 28129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:05.911180 28129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:05.911228 28129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:05.916424 28129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41399
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:05.917429 28129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:05.918421 28129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:05.926831 28137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:05.927000 28129 server_base.cc:1034] running on GCE node
W20260806 12:33:05.926910 28134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:05.927103 28135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:05.927809 28129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:05.929340 28129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:05.930549 28129 hybrid_clock.cc:648] HybridClock initialized: now 1786019585930528 us; error 37 us; skew 500 ppm
Aug 06 12:33:05 dist-test-slave-1xrh krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:05.944051 28129 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:05.945007 28129 webserver.cc:460] Webserver started at http://127.0.0.1:39025/ using document root <none> and password file <none>
I20260806 12:33:05.945567 28129 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:05.945653 28129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:05.945915 28129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:05.948067 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608"
format_stamp: "Formatted at 2026-08-06 12:33:05 on dist-test-slave-1xrh"
server_key: "8dc3d91425bf92785b08e3017b38545a"
server_key_iv: "75566174f509ebd121399022508e5e9a"
server_key_version: "encryptionkey@0"
I20260806 12:33:05.948637 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608"
format_stamp: "Formatted at 2026-08-06 12:33:05 on dist-test-slave-1xrh"
server_key: "8dc3d91425bf92785b08e3017b38545a"
server_key_iv: "75566174f509ebd121399022508e5e9a"
server_key_version: "encryptionkey@0"
I20260806 12:33:05.952313 28129 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:33:05.954576 28144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:05.955802 28129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:33:05.955951 28129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608"
format_stamp: "Formatted at 2026-08-06 12:33:05 on dist-test-slave-1xrh"
server_key: "8dc3d91425bf92785b08e3017b38545a"
server_key_iv: "75566174f509ebd121399022508e5e9a"
server_key_version: "encryptionkey@0"
I20260806 12:33:05.956058 28129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:05.960743 28129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:05.963833 28129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:05.964027 28129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:05.971063 28129 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41399
I20260806 12:33:05.971076 28196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41399 every 8 connection(s)
I20260806 12:33:05.972020 28129 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:05.974601 28197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:05.979919 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28129
I20260806 12:33:05.979945 28197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608: Bootstrap starting.
I20260806 12:33:05.980063 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:05.980352 6754 external_mini_cluster.cc:1428] Setting key a7e9f33e0f95b8527122c92b51127e70
I20260806 12:33:05.982647 28197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:05.983577 28197 log.cc:824] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:05.985498 28197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608: No bootstrap required, opened a new log
I20260806 12:33:05.987067 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:05.981711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50247 (local address 127.0.0.1:41399)
0806 12:33:05.982138 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:05.982156 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:05.982198 (+ 42us) server_negotiation.cc:407] Connection header received
0806 12:33:05.983157 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:05.983180 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:05.983572 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:05.983972 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:05.985881 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:05.985886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:05.985901 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:05.985912 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:05.985964 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:05.985983 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:05.986070 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:05.986223 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 12:33:05.986399 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":101,"threads_started":1}
I20260806 12:33:05.989756 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:05.989622 28197 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41399 } }
I20260806 12:33:05.989861 28197 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:05.989934 28197 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7dfaaeb25b048eba8f9cd1fddb7d608, State: Initialized, Role: FOLLOWER
I20260806 12:33:05.990329 28197 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41399 } }
I20260806 12:33:05.990485 28197 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:05.990566 28197 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:05.990659 28197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:05.991894 28197 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41399 } }
I20260806 12:33:05.992321 28197 leader_election.cc:302] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7dfaaeb25b048eba8f9cd1fddb7d608; no voters:
I20260806 12:33:05.992722 28197 leader_election.cc:288] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:05.993080 28208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:05.993307 28208 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [term 1 LEADER]: Becoming Leader. State: Replica: c7dfaaeb25b048eba8f9cd1fddb7d608, State: Running, Role: LEADER
I20260806 12:33:05.993646 28208 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41399 } }
I20260806 12:33:05.994119 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:05.992082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60081 (local address 127.0.0.1:41399)
0806 12:33:05.992204 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:05.992208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:05.992429 (+ 221us) server_negotiation.cc:407] Connection header received
0806 12:33:05.992557 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:05.992560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:05.992593 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:05.992691 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:05.993510 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:05.993513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:05.993516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:05.993528 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:05.993556 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:05.993570 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:05.993642 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:05.993947 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 12:33:05.993986 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:33:05.993983 28197 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:05.995085 28209 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7dfaaeb25b048eba8f9cd1fddb7d608. Latest consensus state: current_term: 1 leader_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41399 } } }
I20260806 12:33:05.995244 28209 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:05.995997 28207 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dfaaeb25b048eba8f9cd1fddb7d608" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41399 } } }
I20260806 12:33:05.996111 28207 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608 [sys.catalog]: This master's current role is: LEADER
W20260806 12:33:05.999069 28222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:05.999171 28222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:05.999254 28216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:05.999974 28216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:06.005277 28216 catalog_manager.cc:1347] Generated new cluster ID: d45ac4fa0ffd4e6db37bf53464570e57
I20260806 12:33:06.005337 28216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:06.015615 28216 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:06.015875 28216 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:06.021405 28216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7dfaaeb25b048eba8f9cd1fddb7d608: Generated new TSK 0
I20260806 12:33:06.021987 28216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:06.029273 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28129
2026-08-06T12:33:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:06 dist-test-slave-1xrh krb5kdc[28228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:06 dist-test-slave-1xrh krb5kdc[28228](info): set up 2 sockets
Aug 06 12:33:06 dist-test-slave-1xrh krb5kdc[28228](info): commencing operation
krb5kdc: starting...
W20260806 12:33:07.115517 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:07 dist-test-slave-1xrh krb5kdc[28228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:07.242574 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:07.362105 28241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:07.362353 28241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:07.362439 28241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:07.366740 28241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:07.366817 28241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:07.366838 28241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:07.366855 28241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:07.366927 28241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:07.372773 28241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:07.373822 28241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:07.374857 28241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:07.383198 28246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:07.383201 28247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:07.383234 28241 server_base.cc:1034] running on GCE node
W20260806 12:33:07.383193 28249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:07.384047 28241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:07.385042 28241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:07.386204 28241 hybrid_clock.cc:648] HybridClock initialized: now 1786019587386192 us; error 30 us; skew 500 ppm
Aug 06 12:33:07 dist-test-slave-1xrh krb5kdc[28228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019587, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:07.389119 28241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:07.389962 28241 webserver.cc:460] Webserver started at http://127.0.0.1:35637/ using document root <none> and password file <none>
I20260806 12:33:07.390506 28241 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:07.390563 28241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:07.390810 28241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:07.392831 28241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "73d08bbe56284bb09745c4d7e85e7c29"
format_stamp: "Formatted at 2026-08-06 12:33:07 on dist-test-slave-1xrh"
server_key: "181ef7bd243ab004bf608e61f72c5f8f"
server_key_iv: "7ec3d0b086a0940b83ad6dae671b9003"
server_key_version: "encryptionkey@0"
I20260806 12:33:07.393397 28241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "73d08bbe56284bb09745c4d7e85e7c29"
format_stamp: "Formatted at 2026-08-06 12:33:07 on dist-test-slave-1xrh"
server_key: "181ef7bd243ab004bf608e61f72c5f8f"
server_key_iv: "7ec3d0b086a0940b83ad6dae671b9003"
server_key_version: "encryptionkey@0"
I20260806 12:33:07.397050 28241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 12:33:07.399549 28256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:07.400694 28241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:33:07.400812 28241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "73d08bbe56284bb09745c4d7e85e7c29"
format_stamp: "Formatted at 2026-08-06 12:33:07 on dist-test-slave-1xrh"
server_key: "181ef7bd243ab004bf608e61f72c5f8f"
server_key_iv: "7ec3d0b086a0940b83ad6dae671b9003"
server_key_version: "encryptionkey@0"
I20260806 12:33:07.400923 28241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:07.423171 28241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:07.426101 28241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:07.426409 28241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:07.433597 28241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40997
I20260806 12:33:07.433609 28308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40997 every 8 connection(s)
I20260806 12:33:07.434688 28241 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:07.437386 28309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:07.438683 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28241
I20260806 12:33:07.438887 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:07.439335 6754 external_mini_cluster.cc:1428] Setting key 3234dd970e109a2e954aa44bdd0675a5
I20260806 12:33:07.443961 28309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29: Bootstrap starting.
I20260806 12:33:07.446431 28309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:07.447286 28309 log.cc:824] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:33:07 dist-test-slave-1xrh krb5kdc[28228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 12:33:07.449080 28309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29: No bootstrap required, opened a new log
I20260806 12:33:07.453764 28309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d08bbe56284bb09745c4d7e85e7c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40997 } }
I20260806 12:33:07.454011 28309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:07.454067 28309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d08bbe56284bb09745c4d7e85e7c29, State: Initialized, Role: FOLLOWER
I20260806 12:33:07.454085 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:07.440693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53993 (local address 127.0.0.1:40997)
0806 12:33:07.441147 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:07.441168 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:07.441212 (+ 44us) server_negotiation.cc:407] Connection header received
0806 12:33:07.442085 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:07.442108 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:07.442532 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:07.442883 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:07.443654 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:07.445077 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:07.445917 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:07.446620 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:07.446780 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:07.449170 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:07.449176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:07.449191 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:07.449221 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:07.451154 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:07.451602 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:07.451608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:07.451613 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:07.451745 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:07.452006 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:07.452010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:07.452012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:07.452485 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:07.452678 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:07.452860 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 12:33:07.453149 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":112,"threads_started":1}
I20260806 12:33:07.454487 28309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d08bbe56284bb09745c4d7e85e7c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40997 } }
I20260806 12:33:07.454658 28309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:07.454758 28309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:07.454874 28309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:07.455950 28309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d08bbe56284bb09745c4d7e85e7c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40997 } }
I20260806 12:33:07.456291 28309 leader_election.cc:302] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73d08bbe56284bb09745c4d7e85e7c29; no voters:
I20260806 12:33:07.456597 28309 leader_election.cc:288] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:07.456861 28314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:07.457113 28314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [term 1 LEADER]: Becoming Leader. State: Replica: 73d08bbe56284bb09745c4d7e85e7c29, State: Running, Role: LEADER
I20260806 12:33:07.457269 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:07.457734 28309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:07.458065 28314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d08bbe56284bb09745c4d7e85e7c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40997 } }
I20260806 12:33:07.459254 28314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73d08bbe56284bb09745c4d7e85e7c29. Latest consensus state: current_term: 1 leader_uuid: "73d08bbe56284bb09745c4d7e85e7c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d08bbe56284bb09745c4d7e85e7c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40997 } } }
I20260806 12:33:07.459438 28314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:07.460001 28315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73d08bbe56284bb09745c4d7e85e7c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d08bbe56284bb09745c4d7e85e7c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40997 } } }
I20260806 12:33:07.460117 28315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29 [sys.catalog]: This master's current role is: LEADER
W20260806 12:33:07.463141 28335 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:07.463219 28335 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:07.463281 28333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:07.464146 28333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:07.469352 28333 catalog_manager.cc:1347] Generated new cluster ID: c2f53db674e74c90b133eeb30796f7ff
I20260806 12:33:07.469409 28333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:07.472047 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:07.459814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48953 (local address 127.0.0.1:40997)
0806 12:33:07.459917 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:07.459921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:07.460161 (+ 240us) server_negotiation.cc:407] Connection header received
0806 12:33:07.460342 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:07.460348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:07.460400 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:07.460455 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:07.461273 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:07.461799 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:07.462597 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:07.462819 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:07.462930 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:07.468527 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:07.468531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:07.468536 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:07.468562 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:07.470355 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:07.470831 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:07.470834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:07.470836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:07.470914 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:07.471246 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:07.471249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:07.471250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:07.471430 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:07.471548 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:07.471804 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 12:33:07.471908 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 12:33:07.485929 28333 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:07.486718 28333 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:07.490577 28333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73d08bbe56284bb09745c4d7e85e7c29: Generated new TSK 0
I20260806 12:33:07.491194 28333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:07.499225 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28241
2026-08-06T12:33:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:07 dist-test-slave-1xrh krb5kdc[28341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:07 dist-test-slave-1xrh krb5kdc[28341](info): set up 2 sockets
Aug 06 12:33:07 dist-test-slave-1xrh krb5kdc[28341](info): commencing operation
krb5kdc: starting...
W20260806 12:33:08.551788 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:08 dist-test-slave-1xrh krb5kdc[28341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:08.676275 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:08.784368 28362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:08.784680 28362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:08.784754 28362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:08.788761 28362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:08.788861 28362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:08.788901 28362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:08.788939 28362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:08.788971 28362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:08.794130 28362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:08.795239 28362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:08.796288 28362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:08.804551 28367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:08.804569 28368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:08.804535 28370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:08.804844 28362 server_base.cc:1034] running on GCE node
I20260806 12:33:08.805446 28362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:08.806458 28362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:08.807629 28362 hybrid_clock.cc:648] HybridClock initialized: now 1786019588807609 us; error 28 us; skew 500 ppm
Aug 06 12:33:08 dist-test-slave-1xrh krb5kdc[28341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:08.810524 28362 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:08.811522 28362 webserver.cc:460] Webserver started at http://127.0.0.1:34911/ using document root <none> and password file <none>
I20260806 12:33:08.812104 28362 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:08.812199 28362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:08.812434 28362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:08.814440 28362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "106202a5351d4054935e4574c3cca889"
format_stamp: "Formatted at 2026-08-06 12:33:08 on dist-test-slave-1xrh"
server_key: "06442cf650b2499a4f8b6349ec2da981"
server_key_iv: "c18fb25d37df3503c440941f722d0324"
server_key_version: "encryptionkey@0"
I20260806 12:33:08.815014 28362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "106202a5351d4054935e4574c3cca889"
format_stamp: "Formatted at 2026-08-06 12:33:08 on dist-test-slave-1xrh"
server_key: "06442cf650b2499a4f8b6349ec2da981"
server_key_iv: "c18fb25d37df3503c440941f722d0324"
server_key_version: "encryptionkey@0"
I20260806 12:33:08.818683 28362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:33:08.821216 28377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:08.822455 28362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 12:33:08.822592 28362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "106202a5351d4054935e4574c3cca889"
format_stamp: "Formatted at 2026-08-06 12:33:08 on dist-test-slave-1xrh"
server_key: "06442cf650b2499a4f8b6349ec2da981"
server_key_iv: "c18fb25d37df3503c440941f722d0324"
server_key_version: "encryptionkey@0"
I20260806 12:33:08.822692 28362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:08.852371 28362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:08.855749 28362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:08.855923 28362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:08.863435 28362 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35931
I20260806 12:33:08.863464 28429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35931 every 8 connection(s)
I20260806 12:33:08.864367 28362 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:08.866978 28430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:08.872228 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28362
I20260806 12:33:08.872380 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:08.872543 28430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889: Bootstrap starting.
I20260806 12:33:08.872713 6754 external_mini_cluster.cc:1428] Setting key 2c6e06dc7a9863b065a14963c60783ab
I20260806 12:33:08.875006 28430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:08.875743 28430 log.cc:824] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:08.877723 28430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889: No bootstrap required, opened a new log
I20260806 12:33:08.881840 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:08.874102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37623 (local address 127.0.0.1:35931)
0806 12:33:08.874516 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:08.874530 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:08.874565 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:33:08.875304 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:08.875323 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:08.875639 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:08.875961 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:08.876377 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:08.877422 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:08.878659 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:08.879203 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:08.879340 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:08.880134 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:08.880140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:08.880152 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:08.880548 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:08.880598 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:08.880619 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:08.880695 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:08.880874 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 12:33:08.881116 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":117,"threads_started":1}
I20260806 12:33:08.881858 28430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106202a5351d4054935e4574c3cca889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260806 12:33:08.882186 28430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:08.882274 28430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106202a5351d4054935e4574c3cca889, State: Initialized, Role: FOLLOWER
I20260806 12:33:08.882746 28430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106202a5351d4054935e4574c3cca889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260806 12:33:08.882887 28430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:08.882956 28430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:08.883052 28430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:08.884001 28430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106202a5351d4054935e4574c3cca889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260806 12:33:08.884315 28430 leader_election.cc:302] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106202a5351d4054935e4574c3cca889; no voters:
I20260806 12:33:08.884690 28430 leader_election.cc:288] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:08.884756 28435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:08.884770 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:08.884992 28435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [term 1 LEADER]: Becoming Leader. State: Replica: 106202a5351d4054935e4574c3cca889, State: Running, Role: LEADER
I20260806 12:33:08.885272 28435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106202a5351d4054935e4574c3cca889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260806 12:33:08.885929 28430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:08.886379 28435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "106202a5351d4054935e4574c3cca889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106202a5351d4054935e4574c3cca889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260806 12:33:08.886544 28435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:08.886775 28436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106202a5351d4054935e4574c3cca889. Latest consensus state: current_term: 1 leader_uuid: "106202a5351d4054935e4574c3cca889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106202a5351d4054935e4574c3cca889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260806 12:33:08.886973 28436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:08.886992 28446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:08.890264 28446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:08.892201 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:08.887030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46477 (local address 127.0.0.1:35931)
0806 12:33:08.887158 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:08.887163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:08.887711 (+ 548us) server_negotiation.cc:407] Connection header received
0806 12:33:08.887896 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:08.887899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:08.887944 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:08.887992 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:08.888401 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:08.889095 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:08.890191 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:08.890531 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:08.890639 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:08.891556 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:08.891560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:08.891563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:08.891575 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:08.891603 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:08.891619 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:08.891694 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:08.892005 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 12:33:08.892055 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 12:33:08.896657 28446 catalog_manager.cc:1347] Generated new cluster ID: bfc7e51a1204492187f492552faf306b
I20260806 12:33:08.896732 28446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:08.919631 28446 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:08.920873 28446 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:08.936594 28446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 106202a5351d4054935e4574c3cca889: Generated new TSK 0
I20260806 12:33:08.937259 28446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:08.947239 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28362
2026-08-06T12:33:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:09 dist-test-slave-1xrh krb5kdc[28461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:09 dist-test-slave-1xrh krb5kdc[28461](info): set up 2 sockets
Aug 06 12:33:09 dist-test-slave-1xrh krb5kdc[28461](info): commencing operation
krb5kdc: starting...
W20260806 12:33:09.999732 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:10 dist-test-slave-1xrh krb5kdc[28461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:10.122783 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:10.230388 28474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:10.230633 28474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:10.230670 28474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:10.234419 28474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:10.234479 28474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:10.234498 28474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:10.234531 28474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:10.234546 28474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:10.239501 28474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:10.240379 28474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:10.241286 28474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:10.249425 28479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:10.249446 28482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:10.249588 28480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:10.249624 28474 server_base.cc:1034] running on GCE node
I20260806 12:33:10.250305 28474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:10.251408 28474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:10.252584 28474 hybrid_clock.cc:648] HybridClock initialized: now 1786019590252555 us; error 39 us; skew 500 ppm
Aug 06 12:33:10 dist-test-slave-1xrh krb5kdc[28461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:10.255326 28474 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:10.256167 28474 webserver.cc:460] Webserver started at http://127.0.0.1:42233/ using document root <none> and password file <none>
I20260806 12:33:10.256656 28474 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:10.256717 28474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:10.256937 28474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:10.258921 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "738c9859912d4f6995f906cffe2d0e48"
format_stamp: "Formatted at 2026-08-06 12:33:10 on dist-test-slave-1xrh"
server_key: "ce22d24d9182774104861b195d099039"
server_key_iv: "66a831d2a07336a3f0fee4ef60e0e0fe"
server_key_version: "encryptionkey@0"
I20260806 12:33:10.259534 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "738c9859912d4f6995f906cffe2d0e48"
format_stamp: "Formatted at 2026-08-06 12:33:10 on dist-test-slave-1xrh"
server_key: "ce22d24d9182774104861b195d099039"
server_key_iv: "66a831d2a07336a3f0fee4ef60e0e0fe"
server_key_version: "encryptionkey@0"
I20260806 12:33:10.262965 28474 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:33:10.265266 28489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:10.266412 28474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:10.266549 28474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "738c9859912d4f6995f906cffe2d0e48"
format_stamp: "Formatted at 2026-08-06 12:33:10 on dist-test-slave-1xrh"
server_key: "ce22d24d9182774104861b195d099039"
server_key_iv: "66a831d2a07336a3f0fee4ef60e0e0fe"
server_key_version: "encryptionkey@0"
I20260806 12:33:10.266657 28474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:10.271015 28474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:10.273628 28474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:10.273806 28474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:10.280875 28474 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38717
I20260806 12:33:10.280900 28544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38717 every 8 connection(s)
I20260806 12:33:10.281703 28474 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:10.284236 28545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:10.287987 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28474
I20260806 12:33:10.288144 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:10.288406 6754 external_mini_cluster.cc:1428] Setting key e408f867bba85d6b2eac31337723ba13
I20260806 12:33:10.289710 28545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48: Bootstrap starting.
I20260806 12:33:10.291940 28545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:10.292830 28545 log.cc:824] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:10.293443 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:10.289677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52289 (local address 127.0.0.1:38717)
0806 12:33:10.290099 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:10.290113 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:10.290147 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:33:10.290764 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:10.290779 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:10.291036 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:10.291417 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:10.292167 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:10.292175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:10.292192 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:10.292209 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:10.292280 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:10.292306 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:10.292414 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:10.292513 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 12:33:10.292658 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":130,"threads_started":1}
I20260806 12:33:10.294981 28545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48: No bootstrap required, opened a new log
I20260806 12:33:10.296470 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:10.298775 28545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "738c9859912d4f6995f906cffe2d0e48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260806 12:33:10.299037 28545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:10.299101 28545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c9859912d4f6995f906cffe2d0e48, State: Initialized, Role: FOLLOWER
I20260806 12:33:10.299623 28545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "738c9859912d4f6995f906cffe2d0e48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260806 12:33:10.299803 28545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:10.299885 28545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:10.299993 28545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:10.300203 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:10.298446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55345 (local address 127.0.0.1:38717)
0806 12:33:10.298567 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:10.298571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:10.298788 (+ 217us) server_negotiation.cc:407] Connection header received
0806 12:33:10.298856 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:10.298859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:10.298890 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:10.298971 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:10.299719 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:10.299723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:10.299725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:10.299736 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:10.299763 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:10.299775 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:10.299865 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:10.300030 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 12:33:10.300092 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:33:10.301065 28545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "738c9859912d4f6995f906cffe2d0e48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260806 12:33:10.301378 28545 leader_election.cc:302] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 738c9859912d4f6995f906cffe2d0e48; no voters:
I20260806 12:33:10.301622 28545 leader_election.cc:288] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:10.301734 28555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:10.302030 28555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [term 1 LEADER]: Becoming Leader. State: Replica: 738c9859912d4f6995f906cffe2d0e48, State: Running, Role: LEADER
I20260806 12:33:10.302305 28555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "738c9859912d4f6995f906cffe2d0e48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260806 12:33:10.302738 28545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:10.303840 28557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 738c9859912d4f6995f906cffe2d0e48. Latest consensus state: current_term: 1 leader_uuid: "738c9859912d4f6995f906cffe2d0e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "738c9859912d4f6995f906cffe2d0e48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } } }
I20260806 12:33:10.303970 28557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:10.304252 28556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "738c9859912d4f6995f906cffe2d0e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "738c9859912d4f6995f906cffe2d0e48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } } }
I20260806 12:33:10.304352 28564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:10.304412 28556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:10.307492 28564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:10.312541 28564 catalog_manager.cc:1347] Generated new cluster ID: 1ef8d631c050433b80fc95635ccc8ca3
I20260806 12:33:10.312599 28564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:10.328567 28564 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:10.328867 28564 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:10.333963 28564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 738c9859912d4f6995f906cffe2d0e48: Generated new TSK 0
I20260806 12:33:10.334613 28564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:10.342311 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28474
2026-08-06T12:33:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:10 dist-test-slave-1xrh krb5kdc[28576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:10 dist-test-slave-1xrh krb5kdc[28576](info): set up 2 sockets
Aug 06 12:33:10 dist-test-slave-1xrh krb5kdc[28576](info): commencing operation
krb5kdc: starting...
W20260806 12:33:11.410617 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:11 dist-test-slave-1xrh krb5kdc[28576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:11.533653 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:11.641601 28589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:11.641841 28589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:11.641880 28589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:11.645850 28589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:11.645903 28589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:11.645922 28589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:11.645979 28589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:11.646009 28589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:11.651322 28589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:11.652294 28589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:11.653290 28589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:11.661653 28595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:11.661839 28597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:11.661671 28589 server_base.cc:1034] running on GCE node
W20260806 12:33:11.661676 28594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:11.662392 28589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:11.663520 28589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:11.664708 28589 hybrid_clock.cc:648] HybridClock initialized: now 1786019591664677 us; error 43 us; skew 500 ppm
Aug 06 12:33:11 dist-test-slave-1xrh krb5kdc[28576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:11.673053 28589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:11.673913 28589 webserver.cc:460] Webserver started at http://127.0.0.1:34683/ using document root <none> and password file <none>
I20260806 12:33:11.674460 28589 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:11.674553 28589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:11.674785 28589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:11.676847 28589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "cf4ba219d6f9490b83506c468c487663"
format_stamp: "Formatted at 2026-08-06 12:33:11 on dist-test-slave-1xrh"
server_key: "27d59c24c37676f8c77cb6ae315eaf62"
server_key_iv: "f62cbb5803aa76e14129d295658257bd"
server_key_version: "encryptionkey@0"
I20260806 12:33:11.677410 28589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "cf4ba219d6f9490b83506c468c487663"
format_stamp: "Formatted at 2026-08-06 12:33:11 on dist-test-slave-1xrh"
server_key: "27d59c24c37676f8c77cb6ae315eaf62"
server_key_iv: "f62cbb5803aa76e14129d295658257bd"
server_key_version: "encryptionkey@0"
I20260806 12:33:11.681021 28589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 12:33:11.683377 28605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:11.684566 28589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:11.684703 28589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "cf4ba219d6f9490b83506c468c487663"
format_stamp: "Formatted at 2026-08-06 12:33:11 on dist-test-slave-1xrh"
server_key: "27d59c24c37676f8c77cb6ae315eaf62"
server_key_iv: "f62cbb5803aa76e14129d295658257bd"
server_key_version: "encryptionkey@0"
I20260806 12:33:11.684814 28589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:11.696655 28589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:11.699854 28589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:11.700050 28589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:11.707041 28589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34251
I20260806 12:33:11.707095 28664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34251 every 8 connection(s)
I20260806 12:33:11.708031 28589 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:11.709466 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28589
I20260806 12:33:11.709631 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:11.709964 6754 external_mini_cluster.cc:1428] Setting key 0dffb60ee95c5cd2ed569c841b748548
I20260806 12:33:11.710939 28665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:11.717375 28665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663: Bootstrap starting.
Aug 06 12:33:11 dist-test-slave-1xrh krb5kdc[28576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 12:33:11.719838 28665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:11.720546 28665 log.cc:824] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:11.722321 28665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663: No bootstrap required, opened a new log
I20260806 12:33:11.724243 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:11.711401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:35153 (local address 127.0.0.1:34251)
0806 12:33:11.711796 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:11.711815 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:11.711860 (+ 45us) server_negotiation.cc:407] Connection header received
0806 12:33:11.712482 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:11.712497 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:11.712823 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:11.713100 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:11.714007 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:11.715047 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:11.716335 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:11.716779 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:11.716895 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:11.719731 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:11.719736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:11.719749 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:11.719775 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:11.721468 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:11.721991 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:11.721996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:11.722001 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:11.722103 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:11.722453 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:11.722457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:11.722459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:11.722822 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:11.723060 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:11.723343 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 12:33:11.723559 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":141,"threads_started":1}
I20260806 12:33:11.726225 28665 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4ba219d6f9490b83506c468c487663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 12:33:11.726467 28665 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:11.726529 28665 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf4ba219d6f9490b83506c468c487663, State: Initialized, Role: FOLLOWER
I20260806 12:33:11.727049 28665 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4ba219d6f9490b83506c468c487663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 12:33:11.727236 28665 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:11.727272 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:11.727330 28665 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:11.727450 28665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:11.728348 28665 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4ba219d6f9490b83506c468c487663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 12:33:11.728715 28665 leader_election.cc:302] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf4ba219d6f9490b83506c468c487663; no voters:
I20260806 12:33:11.729041 28665 leader_election.cc:288] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:11.729110 28674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:11.729322 28674 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [term 1 LEADER]: Becoming Leader. State: Replica: cf4ba219d6f9490b83506c468c487663, State: Running, Role: LEADER
I20260806 12:33:11.729665 28674 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4ba219d6f9490b83506c468c487663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 12:33:11.730271 28665 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:11.730892 28676 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf4ba219d6f9490b83506c468c487663. Latest consensus state: current_term: 1 leader_uuid: "cf4ba219d6f9490b83506c468c487663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4ba219d6f9490b83506c468c487663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } } }
I20260806 12:33:11.731009 28676 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:11.731411 28675 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf4ba219d6f9490b83506c468c487663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4ba219d6f9490b83506c468c487663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } } }
I20260806 12:33:11.731521 28675 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663 [sys.catalog]: This master's current role is: LEADER
W20260806 12:33:11.735375 28691 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:11.735466 28691 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:11.735575 28681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:11.736411 28681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:11.739274 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:11.729328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:39853 (local address 127.0.0.1:34251)
0806 12:33:11.729437 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:11.729442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:11.729664 (+ 222us) server_negotiation.cc:407] Connection header received
0806 12:33:11.729790 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:11.729796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:11.729842 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:11.730661 (+ 819us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:11.730799 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:11.731594 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:11.733350 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:11.733601 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:11.733724 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:11.735364 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:11.735368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:11.735371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:11.735405 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:11.737199 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:11.737817 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:11.737821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:11.737823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:11.737908 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:11.738320 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:11.738323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:11.738325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:11.738555 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:11.738699 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:11.739005 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 12:33:11.739105 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:33:11.741735 28681 catalog_manager.cc:1347] Generated new cluster ID: 5b5526301870402eb7011c3c902fa651
I20260806 12:33:11.741789 28681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:11.758544 28681 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:11.759863 28681 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:11.771471 28681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf4ba219d6f9490b83506c468c487663: Generated new TSK 0
I20260806 12:33:11.772264 28681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:11.776356 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28589
W20260806 12:33:11.783183 28574 connection.cc:535] client connection to 127.0.0.1:34251 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34251): Connection reset by peer (error 104)
2026-08-06T12:33:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:11 dist-test-slave-1xrh krb5kdc[28697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:11 dist-test-slave-1xrh krb5kdc[28697](info): set up 2 sockets
Aug 06 12:33:11 dist-test-slave-1xrh krb5kdc[28697](info): commencing operation
krb5kdc: starting...
W20260806 12:33:12.854632 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:12 dist-test-slave-1xrh krb5kdc[28697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:12.977918 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:13.087924 28710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:13.088196 28710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:13.088272 28710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:13.092190 28710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:13.092272 28710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:13.092312 28710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:13.092367 28710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:13.092409 28710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:13.097749 28710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:13.098726 28710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:13.099740 28710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:13.107884 28716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:13.107860 28717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:13.107934 28710 server_base.cc:1034] running on GCE node
W20260806 12:33:13.107884 28719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:13.108598 28710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:13.109588 28710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:13.110770 28710 hybrid_clock.cc:648] HybridClock initialized: now 1786019593110747 us; error 34 us; skew 500 ppm
Aug 06 12:33:13 dist-test-slave-1xrh krb5kdc[28697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:13.113569 28710 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:13.114368 28710 webserver.cc:460] Webserver started at http://127.0.0.1:34167/ using document root <none> and password file <none>
I20260806 12:33:13.114905 28710 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:13.114995 28710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:13.115269 28710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:13.117202 28710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "fc81a7842fda4d88b7754cf4a91788b9"
format_stamp: "Formatted at 2026-08-06 12:33:13 on dist-test-slave-1xrh"
server_key: "8b9b41ed34b879990e45b1fa133665e1"
server_key_iv: "1a2ae902b2e06e39db85b5e0aca78be6"
server_key_version: "encryptionkey@0"
I20260806 12:33:13.117756 28710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "fc81a7842fda4d88b7754cf4a91788b9"
format_stamp: "Formatted at 2026-08-06 12:33:13 on dist-test-slave-1xrh"
server_key: "8b9b41ed34b879990e45b1fa133665e1"
server_key_iv: "1a2ae902b2e06e39db85b5e0aca78be6"
server_key_version: "encryptionkey@0"
I20260806 12:33:13.121246 28710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 12:33:13.123463 28726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:13.124574 28710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:13.124678 28710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "fc81a7842fda4d88b7754cf4a91788b9"
format_stamp: "Formatted at 2026-08-06 12:33:13 on dist-test-slave-1xrh"
server_key: "8b9b41ed34b879990e45b1fa133665e1"
server_key_iv: "1a2ae902b2e06e39db85b5e0aca78be6"
server_key_version: "encryptionkey@0"
I20260806 12:33:13.124756 28710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:13.135763 28710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:13.138849 28710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:13.138988 28710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:13.145943 28710 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34517
I20260806 12:33:13.146051 28778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34517 every 8 connection(s)
I20260806 12:33:13.146875 28710 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:13.149500 28779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:13.154165 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28710
I20260806 12:33:13.154311 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:13.154655 6754 external_mini_cluster.cc:1428] Setting key a1b16bc71e9253b3246f9bd0391c4fcb
I20260806 12:33:13.154935 28779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9: Bootstrap starting.
I20260806 12:33:13.157338 28779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:13.158288 28779 log.cc:824] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:13.160473 28779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9: No bootstrap required, opened a new log
I20260806 12:33:13.163821 28782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:13.155983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:38431 (local address 127.0.0.1:34517)
0806 12:33:13.156384 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:13.156405 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:13.156440 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:33:13.157057 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:13.157073 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:13.157398 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:13.157756 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:13.158104 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:13.159400 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:13.160622 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:13.161098 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:13.161218 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:13.162101 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:13.162106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:13.162118 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:13.162501 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:13.162553 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:13.162575 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:13.162675 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:13.162928 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 12:33:13.163068 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":135,"threads_started":1}
I20260806 12:33:13.164544 28779 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc81a7842fda4d88b7754cf4a91788b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34517 } }
I20260806 12:33:13.164778 28779 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:13.164839 28779 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc81a7842fda4d88b7754cf4a91788b9, State: Initialized, Role: FOLLOWER
I20260806 12:33:13.165277 28779 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc81a7842fda4d88b7754cf4a91788b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34517 } }
I20260806 12:33:13.165436 28779 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:13.165503 28779 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:13.165650 28779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:13.166682 28779 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc81a7842fda4d88b7754cf4a91788b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34517 } }
I20260806 12:33:13.166850 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:13.167032 28779 leader_election.cc:302] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc81a7842fda4d88b7754cf4a91788b9; no voters:
I20260806 12:33:13.167353 28779 leader_election.cc:288] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:13.167567 28784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:13.167850 28784 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [term 1 LEADER]: Becoming Leader. State: Replica: fc81a7842fda4d88b7754cf4a91788b9, State: Running, Role: LEADER
I20260806 12:33:13.168246 28784 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc81a7842fda4d88b7754cf4a91788b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34517 } }
I20260806 12:33:13.168695 28779 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:13.169382 28784 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc81a7842fda4d88b7754cf4a91788b9. Latest consensus state: current_term: 1 leader_uuid: "fc81a7842fda4d88b7754cf4a91788b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc81a7842fda4d88b7754cf4a91788b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34517 } } }
I20260806 12:33:13.169556 28784 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:13.169924 28785 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc81a7842fda4d88b7754cf4a91788b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc81a7842fda4d88b7754cf4a91788b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34517 } } }
I20260806 12:33:13.170011 28785 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:13.170044 28798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:13.172788 28798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:13.177975 28798 catalog_manager.cc:1347] Generated new cluster ID: df06e5334df54da9813f1bab6eabc439
I20260806 12:33:13.178090 28798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:13.178431 28782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:13.169323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:33195 (local address 127.0.0.1:34517)
0806 12:33:13.169427 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:13.169432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:13.169573 (+ 141us) server_negotiation.cc:407] Connection header received
0806 12:33:13.169777 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:13.169780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:13.169825 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:13.169879 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:13.170307 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:13.171174 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:13.176251 (+ 5077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:13.176629 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:13.176736 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:13.177843 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:13.177848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:13.177851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:13.177866 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:13.177893 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:13.177906 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:13.178009 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:13.178247 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 12:33:13.178282 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 12:33:13.224052 28798 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:13.224980 28798 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:13.232337 28798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc81a7842fda4d88b7754cf4a91788b9: Generated new TSK 0
I20260806 12:33:13.233016 28798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:13.241909 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28710
W20260806 12:33:13.248951 28695 connection.cc:535] client connection to 127.0.0.1:34517 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34517): Connection reset by peer (error 104)
2026-08-06T12:33:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:13 dist-test-slave-1xrh krb5kdc[28810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:13 dist-test-slave-1xrh krb5kdc[28810](info): set up 2 sockets
Aug 06 12:33:13 dist-test-slave-1xrh krb5kdc[28810](info): commencing operation
krb5kdc: starting...
W20260806 12:33:14.292840 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:14 dist-test-slave-1xrh krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:14.416179 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:14.523190 28824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:14.523420 28824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:14.523460 28824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:14.527448 28824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:14.527500 28824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:14.527519 28824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:14.527535 28824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:14.527580 28824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:14.532579 28824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:14.533540 28824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:14.534492 28824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:14.542732 28830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:14.542689 28832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:14.542704 28824 server_base.cc:1034] running on GCE node
W20260806 12:33:14.542748 28829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:14.543543 28824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:14.544582 28824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:14.545764 28824 hybrid_clock.cc:648] HybridClock initialized: now 1786019594545739 us; error 37 us; skew 500 ppm
Aug 06 12:33:14 dist-test-slave-1xrh krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:14.548583 28824 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:14.549434 28824 webserver.cc:460] Webserver started at http://127.0.0.1:35131/ using document root <none> and password file <none>
I20260806 12:33:14.549976 28824 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:14.550068 28824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:14.550290 28824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:14.552274 28824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "ea052e27b0a6475bb9c2c2133fc2815a"
format_stamp: "Formatted at 2026-08-06 12:33:14 on dist-test-slave-1xrh"
server_key: "4988a88167b26a4d949b654582aeacf7"
server_key_iv: "741592dbaeed4d829ac69b163111bd05"
server_key_version: "encryptionkey@0"
I20260806 12:33:14.552842 28824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "ea052e27b0a6475bb9c2c2133fc2815a"
format_stamp: "Formatted at 2026-08-06 12:33:14 on dist-test-slave-1xrh"
server_key: "4988a88167b26a4d949b654582aeacf7"
server_key_iv: "741592dbaeed4d829ac69b163111bd05"
server_key_version: "encryptionkey@0"
I20260806 12:33:14.556596 28824 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 12:33:14.558950 28839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:14.560107 28824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:33:14.560252 28824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "ea052e27b0a6475bb9c2c2133fc2815a"
format_stamp: "Formatted at 2026-08-06 12:33:14 on dist-test-slave-1xrh"
server_key: "4988a88167b26a4d949b654582aeacf7"
server_key_iv: "741592dbaeed4d829ac69b163111bd05"
server_key_version: "encryptionkey@0"
I20260806 12:33:14.560361 28824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:14.564821 28824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:14.567997 28824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:14.568184 28824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:14.575091 28824 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37623
I20260806 12:33:14.575104 28891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37623 every 8 connection(s)
I20260806 12:33:14.576071 28824 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:14.578679 28892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:14.581334 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28824
I20260806 12:33:14.581490 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:14.581810 6754 external_mini_cluster.cc:1428] Setting key 63a282ab4d984067beb14f6fa88486dd
I20260806 12:33:14.584517 28892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a: Bootstrap starting.
I20260806 12:33:14.586756 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:14.583078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:53885 (local address 127.0.0.1:37623)
0806 12:33:14.583533 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:14.583546 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:14.583579 (+ 33us) server_negotiation.cc:407] Connection header received
0806 12:33:14.584193 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:14.584209 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:14.584483 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:14.584794 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:14.585590 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:14.585595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:14.585606 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:14.585616 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:14.585666 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:14.585684 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:14.585790 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:14.585912 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 12:33:14.586074 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":134,"threads_started":1}
I20260806 12:33:14.587078 28892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:14.587828 28892 log.cc:824] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:14.589437 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:14.589777 28892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a: No bootstrap required, opened a new log
I20260806 12:33:14.593037 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:14.591537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:48869 (local address 127.0.0.1:37623)
0806 12:33:14.591646 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:14.591650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:14.591782 (+ 132us) server_negotiation.cc:407] Connection header received
0806 12:33:14.591904 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:14.591906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:14.591933 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:14.591997 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:14.592655 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:14.592658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:14.592660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:14.592670 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:14.592692 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:14.592702 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:14.592768 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:14.592889 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 12:33:14.592917 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:33:14.593920 28892 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260806 12:33:14.594148 28892 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:14.594213 28892 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea052e27b0a6475bb9c2c2133fc2815a, State: Initialized, Role: FOLLOWER
I20260806 12:33:14.594641 28892 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260806 12:33:14.594787 28892 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:14.594851 28892 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:14.594946 28892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:14.595863 28892 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260806 12:33:14.596190 28892 leader_election.cc:302] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea052e27b0a6475bb9c2c2133fc2815a; no voters:
I20260806 12:33:14.596496 28892 leader_election.cc:288] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:14.596597 28902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:14.596818 28902 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [term 1 LEADER]: Becoming Leader. State: Replica: ea052e27b0a6475bb9c2c2133fc2815a, State: Running, Role: LEADER
I20260806 12:33:14.597157 28902 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260806 12:33:14.597682 28892 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:14.598834 28904 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea052e27b0a6475bb9c2c2133fc2815a. Latest consensus state: current_term: 1 leader_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } } }
I20260806 12:33:14.598970 28904 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:14.599291 28903 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea052e27b0a6475bb9c2c2133fc2815a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } } }
I20260806 12:33:14.599452 28903 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:14.600252 28911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:14.602783 28911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:14.608275 28911 catalog_manager.cc:1347] Generated new cluster ID: 053c2edb706e43d391ee8c2409fc6e61
I20260806 12:33:14.608336 28911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:14.615700 28911 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:14.615984 28911 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:14.622498 28911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea052e27b0a6475bb9c2c2133fc2815a: Generated new TSK 0
I20260806 12:33:14.623201 28911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:14.629189 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28824
2026-08-06T12:33:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:14 dist-test-slave-1xrh krb5kdc[28923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:14 dist-test-slave-1xrh krb5kdc[28923](info): set up 2 sockets
Aug 06 12:33:14 dist-test-slave-1xrh krb5kdc[28923](info): commencing operation
krb5kdc: starting...
W20260806 12:33:15.694926 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:15 dist-test-slave-1xrh krb5kdc[28923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:15.814106 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37979
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:15.919741 28936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:15.919960 28936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:15.919998 28936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:15.923816 28936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:15.923861 28936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:15.923882 28936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:15.923898 28936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:15.923913 28936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:15.928831 28936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:15.929718 28936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:15.930639 28936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:15.938838 28941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:15.939005 28936 server_base.cc:1034] running on GCE node
W20260806 12:33:15.938853 28944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:15.938891 28942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:15.939685 28936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:15.940732 28936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:15.941890 28936 hybrid_clock.cc:648] HybridClock initialized: now 1786019595941863 us; error 38 us; skew 500 ppm
Aug 06 12:33:15 dist-test-slave-1xrh krb5kdc[28923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:15.944664 28936 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:15.945461 28936 webserver.cc:460] Webserver started at http://127.0.0.1:45155/ using document root <none> and password file <none>
I20260806 12:33:15.945969 28936 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:15.946026 28936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:15.946185 28936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:15.948171 28936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "8e0ac56711d84e41b85f455db7fc8852"
format_stamp: "Formatted at 2026-08-06 12:33:15 on dist-test-slave-1xrh"
server_key: "335f32a0407ab3b6f91a09646f60b4c5"
server_key_iv: "935ed9c185d8ad2c446429d80adc09f3"
server_key_version: "encryptionkey@0"
I20260806 12:33:15.948858 28936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "8e0ac56711d84e41b85f455db7fc8852"
format_stamp: "Formatted at 2026-08-06 12:33:15 on dist-test-slave-1xrh"
server_key: "335f32a0407ab3b6f91a09646f60b4c5"
server_key_iv: "935ed9c185d8ad2c446429d80adc09f3"
server_key_version: "encryptionkey@0"
I20260806 12:33:15.954470 28936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 12:33:15.957917 28951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:15.959264 28936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:33:15.959360 28936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "8e0ac56711d84e41b85f455db7fc8852"
format_stamp: "Formatted at 2026-08-06 12:33:15 on dist-test-slave-1xrh"
server_key: "335f32a0407ab3b6f91a09646f60b4c5"
server_key_iv: "935ed9c185d8ad2c446429d80adc09f3"
server_key_version: "encryptionkey@0"
I20260806 12:33:15.959446 28936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:15.984855 28936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:15.987748 28936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:15.987911 28936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:15.995026 28936 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37979
I20260806 12:33:15.995055 29005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37979 every 8 connection(s)
I20260806 12:33:15.996025 28936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:15.998615 29006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:15.999607 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 28936
I20260806 12:33:15.999758 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:16.000099 6754 external_mini_cluster.cc:1428] Setting key 1975188a6a50999cd330234e454a9eef
I20260806 12:33:16.004657 29006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852: Bootstrap starting.
I20260806 12:33:16.006753 29006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:16.007525 29006 log.cc:824] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:33:16 dist-test-slave-1xrh krb5kdc[28923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 12:33:16.009512 29006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852: No bootstrap required, opened a new log
I20260806 12:33:16.012323 29006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0ac56711d84e41b85f455db7fc8852" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37979 } }
I20260806 12:33:16.012544 29006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:16.012632 29006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0ac56711d84e41b85f455db7fc8852, State: Initialized, Role: FOLLOWER
I20260806 12:33:16.013130 29006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0ac56711d84e41b85f455db7fc8852" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37979 } }
I20260806 12:33:16.013262 29006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:16.013339 29006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:16.013448 29006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:16.014292 29006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0ac56711d84e41b85f455db7fc8852" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37979 } }
I20260806 12:33:16.014633 29006 leader_election.cc:302] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e0ac56711d84e41b85f455db7fc8852; no voters:
I20260806 12:33:16.014952 29006 leader_election.cc:288] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:16.015056 29012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:16.015250 29012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [term 1 LEADER]: Becoming Leader. State: Replica: 8e0ac56711d84e41b85f455db7fc8852, State: Running, Role: LEADER
I20260806 12:33:16.015287 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:16.001492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:55173 (local address 127.0.0.1:37979)
0806 12:33:16.001930 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:16.001944 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:16.001979 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:33:16.002613 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:16.002629 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:16.002947 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:16.003286 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:16.004104 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:16.005100 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:16.005963 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:16.006432 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:16.006598 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:16.009455 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:16.009460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:16.009472 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:16.009497 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:16.011963 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:16.012482 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:16.012505 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:16.012509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:16.012723 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:16.013076 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:16.013079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:16.013080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:16.013593 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:16.013837 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:16.014110 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 12:33:16.014318 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":134,"threads_started":1}
I20260806 12:33:16.015560 29012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0ac56711d84e41b85f455db7fc8852" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37979 } }
I20260806 12:33:16.016171 29006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:16.017081 29013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e0ac56711d84e41b85f455db7fc8852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0ac56711d84e41b85f455db7fc8852" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37979 } } }
I20260806 12:33:16.017206 29013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:16.017545 29014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e0ac56711d84e41b85f455db7fc8852. Latest consensus state: current_term: 1 leader_uuid: "8e0ac56711d84e41b85f455db7fc8852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0ac56711d84e41b85f455db7fc8852" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37979 } } }
I20260806 12:33:16.017602 29021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:16.017629 29014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:16.018536 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:16.020932 29021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:16.026451 29021 catalog_manager.cc:1347] Generated new cluster ID: 45ff7d0b2f174c7e83cba826ed97fd24
I20260806 12:33:16.026518 29021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:16.031388 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:16.020835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:38501 (local address 127.0.0.1:37979)
0806 12:33:16.020962 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:16.020966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:16.021266 (+ 300us) server_negotiation.cc:407] Connection header received
0806 12:33:16.021436 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:16.021439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:16.021486 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:16.021536 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:16.022450 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:16.023248 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:16.024448 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:16.024775 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:16.024893 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:16.026497 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:16.026501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:16.026503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:16.026528 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:16.028058 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:16.028528 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:16.028532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:16.028533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:16.028616 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:16.028942 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:16.028945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:16.028947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:16.030320 (+ 1373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:16.030489 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:16.031140 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 12:33:16.031241 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:33:16.045501 29021 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:16.046370 29021 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:16.053148 29021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e0ac56711d84e41b85f455db7fc8852: Generated new TSK 0
I20260806 12:33:16.053792 29021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:16.060258 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 28936
W20260806 12:33:16.066839 28921 connection.cc:535] client connection to 127.0.0.1:37979 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37979): Connection reset by peer (error 104)
2026-08-06T12:33:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:16 dist-test-slave-1xrh krb5kdc[29038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:16 dist-test-slave-1xrh krb5kdc[29038](info): set up 2 sockets
Aug 06 12:33:16 dist-test-slave-1xrh krb5kdc[29038](info): commencing operation
krb5kdc: starting...
W20260806 12:33:17.115600 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:17 dist-test-slave-1xrh krb5kdc[29038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:17.238329 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:17.348506 29051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:17.348811 29051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:17.348901 29051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:17.352872 29051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:17.352952 29051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:17.352988 29051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:17.353044 29051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:17.353087 29051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:17.358247 29051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:17.359367 29051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:17.360463 29051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:17.368723 29056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:17.368746 29059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:17.368724 29057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:17.369019 29051 server_base.cc:1034] running on GCE node
I20260806 12:33:17.369557 29051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:17.370637 29051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:17.371840 29051 hybrid_clock.cc:648] HybridClock initialized: now 1786019597371810 us; error 39 us; skew 500 ppm
Aug 06 12:33:17 dist-test-slave-1xrh krb5kdc[29038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:17.382408 29051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:17.383589 29051 webserver.cc:460] Webserver started at http://127.0.0.1:34147/ using document root <none> and password file <none>
I20260806 12:33:17.384169 29051 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:17.384234 29051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:17.384424 29051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:17.386602 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "2684097c35754388abe55f7b71fff926"
format_stamp: "Formatted at 2026-08-06 12:33:17 on dist-test-slave-1xrh"
server_key: "d5805a0ad3ca480672438982a21715a8"
server_key_iv: "a3ed179f317263932872c470e1a66885"
server_key_version: "encryptionkey@0"
I20260806 12:33:17.387223 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "2684097c35754388abe55f7b71fff926"
format_stamp: "Formatted at 2026-08-06 12:33:17 on dist-test-slave-1xrh"
server_key: "d5805a0ad3ca480672438982a21715a8"
server_key_iv: "a3ed179f317263932872c470e1a66885"
server_key_version: "encryptionkey@0"
I20260806 12:33:17.390991 29051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 12:33:17.393318 29066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:17.394472 29051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:17.394599 29051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "2684097c35754388abe55f7b71fff926"
format_stamp: "Formatted at 2026-08-06 12:33:17 on dist-test-slave-1xrh"
server_key: "d5805a0ad3ca480672438982a21715a8"
server_key_iv: "a3ed179f317263932872c470e1a66885"
server_key_version: "encryptionkey@0"
I20260806 12:33:17.394696 29051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:17.416963 29051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:17.419934 29051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:17.420145 29051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:17.427428 29051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40783
I20260806 12:33:17.427455 29120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40783 every 8 connection(s)
I20260806 12:33:17.428324 29051 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:17.430888 29121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:17.433912 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29051
I20260806 12:33:17.434082 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:17.434401 6754 external_mini_cluster.cc:1428] Setting key ffaa7020f9e0622c5869a3a8883d3f82
I20260806 12:33:17.436774 29121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926: Bootstrap starting.
I20260806 12:33:17.438905 29121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:17.439917 29121 log.cc:824] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:17.442001 29121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926: No bootstrap required, opened a new log
I20260806 12:33:17.444059 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:17.435814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:51333 (local address 127.0.0.1:40783)
0806 12:33:17.436234 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:17.436248 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:17.436300 (+ 52us) server_negotiation.cc:407] Connection header received
0806 12:33:17.436972 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:17.436989 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:17.437309 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:17.437654 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:17.438054 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:17.439254 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:17.440347 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:17.440900 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:17.441089 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:17.442072 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:17.442078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:17.442090 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:17.442523 (+ 433us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:17.442589 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:17.442615 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:17.442957 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:17.443143 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 12:33:17.443333 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":141,"threads_started":1}
I20260806 12:33:17.447124 29121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2684097c35754388abe55f7b71fff926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260806 12:33:17.447497 29121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:17.447568 29121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2684097c35754388abe55f7b71fff926, State: Initialized, Role: FOLLOWER
I20260806 12:33:17.448088 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:17.448155 29121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2684097c35754388abe55f7b71fff926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260806 12:33:17.448313 29121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:17.448377 29121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:17.448472 29121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:17.449697 29121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2684097c35754388abe55f7b71fff926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260806 12:33:17.450124 29121 leader_election.cc:302] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2684097c35754388abe55f7b71fff926; no voters:
I20260806 12:33:17.450503 29121 leader_election.cc:288] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:17.450575 29130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:17.450788 29130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [term 1 LEADER]: Becoming Leader. State: Replica: 2684097c35754388abe55f7b71fff926, State: Running, Role: LEADER
I20260806 12:33:17.451107 29130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2684097c35754388abe55f7b71fff926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } }
I20260806 12:33:17.452000 29121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:17.453437 29132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2684097c35754388abe55f7b71fff926. Latest consensus state: current_term: 1 leader_uuid: "2684097c35754388abe55f7b71fff926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2684097c35754388abe55f7b71fff926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } } }
I20260806 12:33:17.453441 29133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2684097c35754388abe55f7b71fff926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2684097c35754388abe55f7b71fff926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40783 } } }
I20260806 12:33:17.453575 29132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:17.453590 29133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:17.454345 29140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:17.456676 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:17.450611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:57531 (local address 127.0.0.1:40783)
0806 12:33:17.450737 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:17.450741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:17.451065 (+ 324us) server_negotiation.cc:407] Connection header received
0806 12:33:17.451275 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:17.451279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:17.451344 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:17.451418 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:17.451841 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:17.452762 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:17.454138 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:17.454557 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:17.454688 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:17.456023 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:17.456031 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:17.456034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:17.456050 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:17.456081 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:17.456095 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:17.456166 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:17.456452 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 12:33:17.456493 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 12:33:17.457944 29140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:17.463682 29140 catalog_manager.cc:1347] Generated new cluster ID: 69d7ddc163364ef4819d5ae3437b30af
I20260806 12:33:17.463747 29140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:17.492887 29140 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:17.494242 29140 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:17.501181 29140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2684097c35754388abe55f7b71fff926: Generated new TSK 0
I20260806 12:33:17.501843 29140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:17.506829 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29051
W20260806 12:33:17.513469 29036 connection.cc:535] client connection to 127.0.0.1:40783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40783): Connection reset by peer (error 104)
2026-08-06T12:33:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:17 dist-test-slave-1xrh krb5kdc[29153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:17 dist-test-slave-1xrh krb5kdc[29153](info): set up 2 sockets
Aug 06 12:33:17 dist-test-slave-1xrh krb5kdc[29153](info): commencing operation
krb5kdc: starting...
W20260806 12:33:18.583828 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:18 dist-test-slave-1xrh krb5kdc[29153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 12:33:18.709791 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:18.816816 29166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:18.817054 29166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:18.817095 29166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:18.820880 29166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:18.820925 29166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:18.820950 29166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:18.820966 29166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:18.820981 29166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:18.825946 29166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:18.826805 29166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:18.827730 29166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:18.835805 29174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:18.835793 29172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:18.835820 29171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:18.836123 29166 server_base.cc:1034] running on GCE node
I20260806 12:33:18.836658 29166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:18.837764 29166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:18.838949 29166 hybrid_clock.cc:648] HybridClock initialized: now 1786019598838934 us; error 33 us; skew 500 ppm
Aug 06 12:33:18 dist-test-slave-1xrh krb5kdc[29153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:18.841750 29166 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 12:33:18.842592 29166 webserver.cc:460] Webserver started at http://127.0.0.1:34247/ using document root <none> and password file <none>
I20260806 12:33:18.843194 29166 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:18.843283 29166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:18.843514 29166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:18.845508 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "badd9ff0ae224b51aa800f60d9518a66"
format_stamp: "Formatted at 2026-08-06 12:33:18 on dist-test-slave-1xrh"
server_key: "e923c91d3b815018a464c15e5049964c"
server_key_iv: "7429f33395b6ebe8a437e93b1b3c58c2"
server_key_version: "encryptionkey@0"
I20260806 12:33:18.846061 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "badd9ff0ae224b51aa800f60d9518a66"
format_stamp: "Formatted at 2026-08-06 12:33:18 on dist-test-slave-1xrh"
server_key: "e923c91d3b815018a464c15e5049964c"
server_key_iv: "7429f33395b6ebe8a437e93b1b3c58c2"
server_key_version: "encryptionkey@0"
I20260806 12:33:18.849634 29166 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 12:33:18.852017 29181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:18.853184 29166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:18.853322 29166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "badd9ff0ae224b51aa800f60d9518a66"
format_stamp: "Formatted at 2026-08-06 12:33:18 on dist-test-slave-1xrh"
server_key: "e923c91d3b815018a464c15e5049964c"
server_key_iv: "7429f33395b6ebe8a437e93b1b3c58c2"
server_key_version: "encryptionkey@0"
I20260806 12:33:18.853430 29166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:18.857816 29166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:18.860620 29166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:18.860805 29166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:18.868094 29166 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40123
I20260806 12:33:18.868108 29233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40123 every 8 connection(s)
I20260806 12:33:18.868943 29166 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:18.871510 29234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:18.875494 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29166
I20260806 12:33:18.875636 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:18.875976 6754 external_mini_cluster.cc:1428] Setting key c309e33711ab7a328e4eeb747a63bc66
I20260806 12:33:18.877157 29234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66: Bootstrap starting.
I20260806 12:33:18.879276 29234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:18.880095 29234 log.cc:824] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:18.880867 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:18.877339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:42825 (local address 127.0.0.1:40123)
0806 12:33:18.877757 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:18.877771 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:18.877805 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:33:18.878405 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:18.878421 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:18.878678 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:18.879004 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:18.879753 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:18.879757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:18.879769 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:18.879779 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:18.879828 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:18.879846 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:18.879969 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:18.880069 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 12:33:18.880222 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":151,"threads_started":1}
I20260806 12:33:18.881920 29234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66: No bootstrap required, opened a new log
I20260806 12:33:18.883514 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:18.885718 29234 raft_consensus.cc:348] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badd9ff0ae224b51aa800f60d9518a66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260806 12:33:18.885989 29234 raft_consensus.cc:374] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:18.886077 29234 raft_consensus.cc:729] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: badd9ff0ae224b51aa800f60d9518a66, State: Initialized, Role: FOLLOWER
I20260806 12:33:18.886533 29234 consensus_queue.cc:260] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badd9ff0ae224b51aa800f60d9518a66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260806 12:33:18.886698 29234 raft_consensus.cc:388] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:18.886775 29234 raft_consensus.cc:482] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:18.886880 29234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:18.887356 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:18.885511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:46311 (local address 127.0.0.1:40123)
0806 12:33:18.885631 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:18.885635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:18.885895 (+ 260us) server_negotiation.cc:407] Connection header received
0806 12:33:18.886018 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:18.886021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:18.886054 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:18.886151 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:18.886829 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:18.886834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:18.886840 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:18.886853 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:18.886878 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:18.886890 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:18.886999 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:18.887116 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 12:33:18.887218 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:33:18.887951 29234 raft_consensus.cc:504] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badd9ff0ae224b51aa800f60d9518a66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260806 12:33:18.888304 29234 leader_election.cc:302] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: badd9ff0ae224b51aa800f60d9518a66; no voters:
I20260806 12:33:18.888582 29234 leader_election.cc:288] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:18.888691 29245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:18.888963 29245 raft_consensus.cc:686] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [term 1 LEADER]: Becoming Leader. State: Replica: badd9ff0ae224b51aa800f60d9518a66, State: Running, Role: LEADER
I20260806 12:33:18.889230 29245 consensus_queue.cc:237] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badd9ff0ae224b51aa800f60d9518a66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } }
I20260806 12:33:18.889710 29234 sys_catalog.cc:564] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:18.890825 29247 sys_catalog.cc:455] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader badd9ff0ae224b51aa800f60d9518a66. Latest consensus state: current_term: 1 leader_uuid: "badd9ff0ae224b51aa800f60d9518a66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badd9ff0ae224b51aa800f60d9518a66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } } }
I20260806 12:33:18.890956 29247 sys_catalog.cc:458] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:18.891274 29246 sys_catalog.cc:455] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "badd9ff0ae224b51aa800f60d9518a66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "badd9ff0ae224b51aa800f60d9518a66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40123 } } }
I20260806 12:33:18.891451 29246 sys_catalog.cc:458] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66 [sys.catalog]: This master's current role is: LEADER
W20260806 12:33:18.894145 29260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:18.894209 29260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:18.894290 29254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:18.895022 29254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:18.900063 29254 catalog_manager.cc:1347] Generated new cluster ID: 4af4469d2bc8404baeff041cc3e5a996
I20260806 12:33:18.900122 29254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:18.918813 29254 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:18.919234 29254 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:18.924793 29254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P badd9ff0ae224b51aa800f60d9518a66: Generated new TSK 0
I20260806 12:33:18.925470 29254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:18.932552 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29166
2026-08-06T12:33:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:18 dist-test-slave-1xrh krb5kdc[29266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:18 dist-test-slave-1xrh krb5kdc[29266](info): set up 2 sockets
Aug 06 12:33:18 dist-test-slave-1xrh krb5kdc[29266](info): commencing operation
krb5kdc: starting...
W20260806 12:33:19.992097 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:20 dist-test-slave-1xrh krb5kdc[29266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:20.117741 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45613
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:45613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:20.225653 29279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:20.225972 29279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:20.226049 29279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:20.230031 29279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:20.230123 29279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:20.230161 29279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:20.230219 29279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:20.230260 29279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:20.235424 29279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:45613
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:20.236537 29279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:20.237556 29279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:20.246021 29287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:20.246054 29284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:20.246057 29285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:20.246095 29279 server_base.cc:1034] running on GCE node
I20260806 12:33:20.246796 29279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:20.247880 29279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:20.249037 29279 hybrid_clock.cc:648] HybridClock initialized: now 1786019600249020 us; error 30 us; skew 500 ppm
Aug 06 12:33:20 dist-test-slave-1xrh krb5kdc[29266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019600, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:20.251708 29279 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:20.252506 29279 webserver.cc:460] Webserver started at http://127.6.152.190:34583/ using document root <none> and password file <none>
I20260806 12:33:20.253090 29279 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:20.253154 29279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:20.253312 29279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:20.255332 29279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "6b9bf620049f4738a059a3785ec59afe"
format_stamp: "Formatted at 2026-08-06 12:33:20 on dist-test-slave-1xrh"
server_key: "fa7bcf23961736e1ca3790c388bed27b"
server_key_iv: "877f228dfb8c6b69c0a14ef84dd37e71"
server_key_version: "encryptionkey@0"
I20260806 12:33:20.255849 29279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "6b9bf620049f4738a059a3785ec59afe"
format_stamp: "Formatted at 2026-08-06 12:33:20 on dist-test-slave-1xrh"
server_key: "fa7bcf23961736e1ca3790c388bed27b"
server_key_iv: "877f228dfb8c6b69c0a14ef84dd37e71"
server_key_version: "encryptionkey@0"
I20260806 12:33:20.259678 29279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 12:33:20.261828 29294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:20.263036 29279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:20.263182 29279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "6b9bf620049f4738a059a3785ec59afe"
format_stamp: "Formatted at 2026-08-06 12:33:20 on dist-test-slave-1xrh"
server_key: "fa7bcf23961736e1ca3790c388bed27b"
server_key_iv: "877f228dfb8c6b69c0a14ef84dd37e71"
server_key_version: "encryptionkey@0"
I20260806 12:33:20.263311 29279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:20.285259 29279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:20.287901 29279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:20.288084 29279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:20.295043 29279 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:45613
I20260806 12:33:20.295059 29346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:45613 every 8 connection(s)
I20260806 12:33:20.296046 29279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:20.298575 29347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:20.303387 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29279
I20260806 12:33:20.303524 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:20.303838 6754 external_mini_cluster.cc:1428] Setting key d051e509bc3d1ccbe01dbae9a294f851
I20260806 12:33:20.303822 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe: Bootstrap starting.
I20260806 12:33:20.306149 29347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:20.306903 29347 log.cc:824] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:20.308808 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe: No bootstrap required, opened a new log
Aug 06 12:33:20 dist-test-slave-1xrh krb5kdc[29266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:20.313163 29347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9bf620049f4738a059a3785ec59afe" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45613 } }
I20260806 12:33:20.313386 29347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:20.313472 29347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9bf620049f4738a059a3785ec59afe, State: Initialized, Role: FOLLOWER
I20260806 12:33:20.313926 29347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9bf620049f4738a059a3785ec59afe" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45613 } }
I20260806 12:33:20.314073 29347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:20.314160 29347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:20.314257 29347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:20.315062 29347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9bf620049f4738a059a3785ec59afe" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45613 } }
I20260806 12:33:20.315410 29347 leader_election.cc:302] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b9bf620049f4738a059a3785ec59afe; no voters:
I20260806 12:33:20.315733 29347 leader_election.cc:288] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:20.315796 29352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:20.316002 29352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [term 1 LEADER]: Becoming Leader. State: Replica: 6b9bf620049f4738a059a3785ec59afe, State: Running, Role: LEADER
I20260806 12:33:20.316279 29352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9bf620049f4738a059a3785ec59afe" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45613 } }
I20260806 12:33:20.316953 29347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:20.318361 29353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b9bf620049f4738a059a3785ec59afe. Latest consensus state: current_term: 1 leader_uuid: "6b9bf620049f4738a059a3785ec59afe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9bf620049f4738a059a3785ec59afe" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45613 } } }
I20260806 12:33:20.318755 29353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:20.318951 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:20.305163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35227 (local address 127.6.152.190:45613)
0806 12:33:20.305562 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:20.305578 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:20.305612 (+ 34us) server_negotiation.cc:407] Connection header received
0806 12:33:20.306200 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:20.306214 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:20.306577 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:20.306833 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:20.307782 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:20.308765 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:20.309911 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:20.310385 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:20.310510 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:20.313349 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:20.313355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:20.313368 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:20.313416 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:20.315393 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:20.315964 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:20.315969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:20.315973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:20.316204 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:20.316654 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:20.316659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:20.316666 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:20.317055 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:20.317288 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:20.317620 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 12:33:20.317852 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260806 12:33:20.318647 29354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b9bf620049f4738a059a3785ec59afe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9bf620049f4738a059a3785ec59afe" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45613 } } }
I20260806 12:33:20.319327 29354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:20.319621 29367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:20.321995 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:20.322654 29367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:20.328449 29367 catalog_manager.cc:1347] Generated new cluster ID: 39e836efb85a422d8b2f72a71c13c89b
I20260806 12:33:20.328513 29367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:20.332487 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:20.324291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51401 (local address 127.6.152.190:45613)
0806 12:33:20.324408 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:20.324412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:20.324614 (+ 202us) server_negotiation.cc:407] Connection header received
0806 12:33:20.324731 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:20.324734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:20.324773 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:20.324864 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:20.325499 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:20.326008 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:20.326873 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:20.327265 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:20.327366 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:20.329093 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:20.329097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:20.329100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:20.329124 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:20.330690 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:20.331249 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:20.331253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:20.331254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:20.331329 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:20.331679 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:20.331682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:20.331683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:20.331863 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:20.331993 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:20.332270 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 12:33:20.332353 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:33:20.347327 29367 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:20.348495 29367 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:20.359628 29367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b9bf620049f4738a059a3785ec59afe: Generated new TSK 0
I20260806 12:33:20.360407 29367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:20.368265 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29279
W20260806 12:33:20.375180 29264 connection.cc:535] client connection to 127.6.152.190:45613 recv error: Network error: failed to read from TLS socket (remote: 127.6.152.190:45613): Connection reset by peer (error 104)
2026-08-06T12:33:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:20 dist-test-slave-1xrh krb5kdc[29380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:20 dist-test-slave-1xrh krb5kdc[29380](info): set up 2 sockets
Aug 06 12:33:20 dist-test-slave-1xrh krb5kdc[29380](info): commencing operation
krb5kdc: starting...
W20260806 12:33:21.419575 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:21 dist-test-slave-1xrh krb5kdc[29380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:21.539342 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35375
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:42101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:35375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:21.647852 29393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:21.648111 29393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:21.648181 29393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:21.652076 29393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:21.652154 29393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:21.652194 29393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:21.652249 29393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:21.652288 29393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:21.657305 29393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:42101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:35375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:35375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:21.658270 29393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:21.659297 29393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:21.667739 29401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:21.667739 29399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:21.667809 29398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:21.668213 29393 server_base.cc:1034] running on GCE node
I20260806 12:33:21.668673 29393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:21.669781 29393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:21.670974 29393 hybrid_clock.cc:648] HybridClock initialized: now 1786019601670944 us; error 41 us; skew 500 ppm
Aug 06 12:33:21 dist-test-slave-1xrh krb5kdc[29380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:21.673621 29393 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:21.674397 29393 webserver.cc:460] Webserver started at http://127.6.152.190:37779/ using document root <none> and password file <none>
I20260806 12:33:21.674880 29393 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:21.674937 29393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:21.675109 29393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:21.677057 29393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "d3becfc74fcc432fbe993a7beacad989"
format_stamp: "Formatted at 2026-08-06 12:33:21 on dist-test-slave-1xrh"
server_key: "fb79c0844d1718f8ab0fe2ae78380f9c"
server_key_iv: "46b2cb14eb2c403cdb4ed16726161571"
server_key_version: "encryptionkey@0"
I20260806 12:33:21.677551 29393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "d3becfc74fcc432fbe993a7beacad989"
format_stamp: "Formatted at 2026-08-06 12:33:21 on dist-test-slave-1xrh"
server_key: "fb79c0844d1718f8ab0fe2ae78380f9c"
server_key_iv: "46b2cb14eb2c403cdb4ed16726161571"
server_key_version: "encryptionkey@0"
I20260806 12:33:21.681058 29393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 12:33:21.683306 29408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:21.684484 29393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 12:33:21.684579 29393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "d3becfc74fcc432fbe993a7beacad989"
format_stamp: "Formatted at 2026-08-06 12:33:21 on dist-test-slave-1xrh"
server_key: "fb79c0844d1718f8ab0fe2ae78380f9c"
server_key_iv: "46b2cb14eb2c403cdb4ed16726161571"
server_key_version: "encryptionkey@0"
I20260806 12:33:21.684659 29393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:21.700431 29393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:21.703083 29393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:21.703315 29393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:21.710017 29393 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:35375
I20260806 12:33:21.710110 29462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:35375 every 8 connection(s)
I20260806 12:33:21.710934 29393 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:21.713655 29463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:21.714876 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29393
I20260806 12:33:21.715034 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:21.715370 6754 external_mini_cluster.cc:1428] Setting key d153eaae673d32d28125c884521225b6
I20260806 12:33:21.720306 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989: Bootstrap starting.
I20260806 12:33:21.722755 29463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:21.723685 29463 log.cc:824] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:21.725026 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:21.716740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33365 (local address 127.6.152.190:35375)
0806 12:33:21.717146 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:21.717164 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:21.717207 (+ 43us) server_negotiation.cc:407] Connection header received
0806 12:33:21.718014 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:21.718036 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:21.718468 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:21.718811 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:21.719257 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:21.720657 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:21.721516 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:21.722171 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:21.722321 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:21.723167 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:21.723173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:21.723189 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:21.723671 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:21.723737 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:21.723771 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:21.723925 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:21.724117 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 12:33:21.724253 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":106,"threads_started":1}
I20260806 12:33:21.725731 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989: No bootstrap required, opened a new log
I20260806 12:33:21.727778 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:21.729460 29463 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3becfc74fcc432fbe993a7beacad989" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35375 } }
I20260806 12:33:21.729792 29463 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:21.729899 29463 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3becfc74fcc432fbe993a7beacad989, State: Initialized, Role: FOLLOWER
I20260806 12:33:21.730412 29463 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3becfc74fcc432fbe993a7beacad989" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35375 } }
I20260806 12:33:21.730611 29463 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:21.730690 29463 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:21.730784 29463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:21.731724 29463 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3becfc74fcc432fbe993a7beacad989" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35375 } }
I20260806 12:33:21.732046 29463 leader_election.cc:302] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3becfc74fcc432fbe993a7beacad989; no voters:
I20260806 12:33:21.732366 29463 leader_election.cc:288] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:21.732450 29473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:21.732623 29473 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [term 1 LEADER]: Becoming Leader. State: Replica: d3becfc74fcc432fbe993a7beacad989, State: Running, Role: LEADER
I20260806 12:33:21.732894 29473 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3becfc74fcc432fbe993a7beacad989" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35375 } }
I20260806 12:33:21.733579 29463 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:21.734016 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:21.729794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49959 (local address 127.6.152.190:35375)
0806 12:33:21.729924 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:21.729928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:21.730205 (+ 277us) server_negotiation.cc:407] Connection header received
0806 12:33:21.730318 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:21.730320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:21.730364 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:21.730446 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:21.730672 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:21.731320 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:21.732193 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:21.732506 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:21.732610 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:21.733506 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:21.733509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:21.733512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:21.733524 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:21.733555 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:21.733566 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:21.733661 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:21.733839 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 12:33:21.733872 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:33:21.733994 29473 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3becfc74fcc432fbe993a7beacad989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3becfc74fcc432fbe993a7beacad989" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35375 } } }
I20260806 12:33:21.734091 29473 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:21.734274 29473 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3becfc74fcc432fbe993a7beacad989. Latest consensus state: current_term: 1 leader_uuid: "d3becfc74fcc432fbe993a7beacad989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3becfc74fcc432fbe993a7beacad989" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 35375 } } }
I20260806 12:33:21.734452 29473 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:21.734983 29477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:21.737739 29477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:21.743060 29477 catalog_manager.cc:1347] Generated new cluster ID: 6c246456ad6e4bc48c8f44783d14f9e9
I20260806 12:33:21.743130 29477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:21.753454 29477 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:21.754294 29477 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:21.758836 29477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3becfc74fcc432fbe993a7beacad989: Generated new TSK 0
I20260806 12:33:21.759518 29477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:21.767318 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29393
W20260806 12:33:21.774112 29378 connection.cc:535] client connection to 127.6.152.190:35375 recv error: Network error: failed to read from TLS socket (remote: 127.6.152.190:35375): Connection reset by peer (error 104)
2026-08-06T12:33:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:21 dist-test-slave-1xrh krb5kdc[29495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:21 dist-test-slave-1xrh krb5kdc[29495](info): set up 2 sockets
Aug 06 12:33:21 dist-test-slave-1xrh krb5kdc[29495](info): commencing operation
krb5kdc: starting...
W20260806 12:33:22.827373 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:22 dist-test-slave-1xrh krb5kdc[29495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:22.960192 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:43947
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:23.071415 29509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:23.071662 29509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:23.071738 29509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:23.075861 29509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:23.075925 29509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:23.075945 29509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:23.075963 29509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:23.076025 29509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:23.081169 29509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:43947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:43947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:23.082154 29509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:23.083185 29509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:23.091454 29517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:23.091456 29514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:23.091483 29515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:23.091647 29509 server_base.cc:1034] running on GCE node
I20260806 12:33:23.092324 29509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:23.093457 29509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:23.094682 29509 hybrid_clock.cc:648] HybridClock initialized: now 1786019603094657 us; error 32 us; skew 500 ppm
Aug 06 12:33:23 dist-test-slave-1xrh krb5kdc[29495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019603, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:23.097929 29509 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:23.098840 29509 webserver.cc:460] Webserver started at http://127.6.152.190:41143/ using document root <none> and password file <none>
I20260806 12:33:23.099474 29509 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:23.099545 29509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:23.099720 29509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:23.101863 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "56a89ec412314bfc98af6aa2bc4612a5"
format_stamp: "Formatted at 2026-08-06 12:33:23 on dist-test-slave-1xrh"
server_key: "c41836050293ab0f329ff4438fef67d2"
server_key_iv: "73281ac2e6e22d0006ad3477fc315807"
server_key_version: "encryptionkey@0"
I20260806 12:33:23.102427 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "56a89ec412314bfc98af6aa2bc4612a5"
format_stamp: "Formatted at 2026-08-06 12:33:23 on dist-test-slave-1xrh"
server_key: "c41836050293ab0f329ff4438fef67d2"
server_key_iv: "73281ac2e6e22d0006ad3477fc315807"
server_key_version: "encryptionkey@0"
I20260806 12:33:23.106308 29509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 12:33:23.108757 29524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:23.110006 29509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 12:33:23.110108 29509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "56a89ec412314bfc98af6aa2bc4612a5"
format_stamp: "Formatted at 2026-08-06 12:33:23 on dist-test-slave-1xrh"
server_key: "c41836050293ab0f329ff4438fef67d2"
server_key_iv: "73281ac2e6e22d0006ad3477fc315807"
server_key_version: "encryptionkey@0"
I20260806 12:33:23.110194 29509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:23.114531 29509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:23.117741 29509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:23.117879 29509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:23.124724 29509 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:43947
I20260806 12:33:23.124738 29577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:43947 every 8 connection(s)
I20260806 12:33:23.125653 29509 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:23.126166 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29509
I20260806 12:33:23.126318 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:23.126608 6754 external_mini_cluster.cc:1428] Setting key ee321c2f28b9812518b5de69a5c54df8
I20260806 12:33:23.128592 29578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:23.131589 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:23.127953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48885 (local address 127.6.152.190:43947)
0806 12:33:23.128362 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:23.128376 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:23.128416 (+ 40us) server_negotiation.cc:407] Connection header received
0806 12:33:23.129058 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:23.129075 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:23.129335 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:23.129603 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:23.130400 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:23.130405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:23.130416 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:23.130426 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:23.130480 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:23.130498 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:23.130601 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:23.130709 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 12:33:23.130861 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":145,"threads_started":1}
I20260806 12:33:23.134321 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:23.134748 29578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5: Bootstrap starting.
I20260806 12:33:23.137463 29578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:23.138218 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:23.136596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40709 (local address 127.6.152.190:43947)
0806 12:33:23.136721 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:23.136725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:23.136890 (+ 165us) server_negotiation.cc:407] Connection header received
0806 12:33:23.137033 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:23.137036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:23.137065 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:23.137120 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:23.137842 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:23.137845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:23.137847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:23.137857 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:23.137879 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:23.137889 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:23.137952 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:23.138064 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 12:33:23.138091 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:33:23.138406 29578 log.cc:824] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:23.140271 29578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5: No bootstrap required, opened a new log
I20260806 12:33:23.143714 29578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a89ec412314bfc98af6aa2bc4612a5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43947 } }
I20260806 12:33:23.143991 29578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:23.144112 29578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a89ec412314bfc98af6aa2bc4612a5, State: Initialized, Role: FOLLOWER
I20260806 12:33:23.144555 29578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a89ec412314bfc98af6aa2bc4612a5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43947 } }
I20260806 12:33:23.144696 29578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:23.144761 29578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:23.144863 29578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:23.145741 29578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a89ec412314bfc98af6aa2bc4612a5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43947 } }
I20260806 12:33:23.146072 29578 leader_election.cc:302] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a89ec412314bfc98af6aa2bc4612a5; no voters:
I20260806 12:33:23.146356 29578 leader_election.cc:288] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:23.146469 29588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:23.146724 29588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [term 1 LEADER]: Becoming Leader. State: Replica: 56a89ec412314bfc98af6aa2bc4612a5, State: Running, Role: LEADER
I20260806 12:33:23.147083 29588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a89ec412314bfc98af6aa2bc4612a5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43947 } }
I20260806 12:33:23.147630 29578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:23.148811 29589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a89ec412314bfc98af6aa2bc4612a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a89ec412314bfc98af6aa2bc4612a5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43947 } } }
I20260806 12:33:23.148931 29589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:23.149312 29597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:23.149295 29590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a89ec412314bfc98af6aa2bc4612a5. Latest consensus state: current_term: 1 leader_uuid: "56a89ec412314bfc98af6aa2bc4612a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a89ec412314bfc98af6aa2bc4612a5" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 43947 } } }
I20260806 12:33:23.149518 29590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:23.152324 29597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:23.157548 29597 catalog_manager.cc:1347] Generated new cluster ID: 53fdc4fdd3f34561ae15571c78111dec
I20260806 12:33:23.157617 29597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:23.190133 29597 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:23.190474 29597 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:23.197389 29597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a89ec412314bfc98af6aa2bc4612a5: Generated new TSK 0
I20260806 12:33:23.198027 29597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:23.204512 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29509
2026-08-06T12:33:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:23 dist-test-slave-1xrh krb5kdc[29609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:23 dist-test-slave-1xrh krb5kdc[29609](info): set up 2 sockets
Aug 06 12:33:23 dist-test-slave-1xrh krb5kdc[29609](info): commencing operation
krb5kdc: starting...
W20260806 12:33:24.264484 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:24 dist-test-slave-1xrh krb5kdc[29609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:24.386675 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:42235
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:42235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:24.497537 29623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:24.497764 29623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:24.497802 29623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:24.501663 29623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:24.501714 29623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:24.501734 29623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:24.501749 29623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:24.501763 29623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:24.506732 29623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:42235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:24.507678 29623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:24.508551 29623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:24.516738 29630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:24.516744 29628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:24.516749 29632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:24.517073 29623 server_base.cc:1034] running on GCE node
I20260806 12:33:24.517692 29623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:24.518716 29623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:24.519873 29623 hybrid_clock.cc:648] HybridClock initialized: now 1786019604519854 us; error 30 us; skew 500 ppm
Aug 06 12:33:24 dist-test-slave-1xrh krb5kdc[29609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019604, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:24.522576 29623 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:24.523428 29623 webserver.cc:460] Webserver started at http://127.6.152.190:34003/ using document root <none> and password file <none>
I20260806 12:33:24.523973 29623 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:24.524034 29623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:24.524278 29623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:24.526235 29623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "354f46db004846c1bda42c45a1135415"
format_stamp: "Formatted at 2026-08-06 12:33:24 on dist-test-slave-1xrh"
server_key: "03e3df4fb44d29f37e76a8604f6911ef"
server_key_iv: "d321e7c9836bfb84d9cd78eca4a60ea9"
server_key_version: "encryptionkey@0"
I20260806 12:33:24.526847 29623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "354f46db004846c1bda42c45a1135415"
format_stamp: "Formatted at 2026-08-06 12:33:24 on dist-test-slave-1xrh"
server_key: "03e3df4fb44d29f37e76a8604f6911ef"
server_key_iv: "d321e7c9836bfb84d9cd78eca4a60ea9"
server_key_version: "encryptionkey@0"
I20260806 12:33:24.530454 29623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:33:24.532734 29639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:24.533797 29623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 12:33:24.533938 29623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "354f46db004846c1bda42c45a1135415"
format_stamp: "Formatted at 2026-08-06 12:33:24 on dist-test-slave-1xrh"
server_key: "03e3df4fb44d29f37e76a8604f6911ef"
server_key_iv: "d321e7c9836bfb84d9cd78eca4a60ea9"
server_key_version: "encryptionkey@0"
I20260806 12:33:24.534029 29623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:24.565546 29623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:24.568794 29623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:24.568957 29623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:24.575937 29623 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:42235
I20260806 12:33:24.576014 29692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:42235 every 8 connection(s)
I20260806 12:33:24.576874 29623 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:24.579501 29693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:24.582849 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29623
I20260806 12:33:24.583001 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:24.583395 6754 external_mini_cluster.cc:1428] Setting key 29c9f5659e6703d9545c824a65433bc5
I20260806 12:33:24.585279 29693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415: Bootstrap starting.
I20260806 12:33:24.587692 29693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:24.588387 29693 log.cc:824] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:24.590471 29693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415: No bootstrap required, opened a new log
Aug 06 12:33:24 dist-test-slave-1xrh krb5kdc[29609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:24.594017 29693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354f46db004846c1bda42c45a1135415" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42235 } }
I20260806 12:33:24.594271 29693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:24.594363 29693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354f46db004846c1bda42c45a1135415, State: Initialized, Role: FOLLOWER
I20260806 12:33:24.594841 29693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354f46db004846c1bda42c45a1135415" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42235 } }
I20260806 12:33:24.594998 29693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:24.595074 29693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:24.595216 29693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:24.596118 29693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354f46db004846c1bda42c45a1135415" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42235 } }
I20260806 12:33:24.596469 29693 leader_election.cc:302] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354f46db004846c1bda42c45a1135415; no voters:
I20260806 12:33:24.596882 29693 leader_election.cc:288] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:24.596976 29698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:24.597187 29698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [term 1 LEADER]: Becoming Leader. State: Replica: 354f46db004846c1bda42c45a1135415, State: Running, Role: LEADER
I20260806 12:33:24.597568 29698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354f46db004846c1bda42c45a1135415" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42235 } }
I20260806 12:33:24.597885 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:24.584706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59947 (local address 127.6.152.190:42235)
0806 12:33:24.585103 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:24.585124 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:24.585159 (+ 35us) server_negotiation.cc:407] Connection header received
0806 12:33:24.585833 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:24.585850 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:24.586174 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:24.586440 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:24.587342 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:24.588339 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:24.589424 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:24.589871 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:24.589994 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:24.592666 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:24.592671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:24.592685 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:24.592712 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:24.594595 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:24.595048 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:24.595055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:24.595061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:24.595226 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:24.595523 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:24.595528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:24.595529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:24.595922 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:24.596125 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:24.596479 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 12:33:24.596810 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":130,"threads_started":1}
I20260806 12:33:24.598179 29693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:24.598796 29698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354f46db004846c1bda42c45a1135415. Latest consensus state: current_term: 1 leader_uuid: "354f46db004846c1bda42c45a1135415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354f46db004846c1bda42c45a1135415" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42235 } } }
I20260806 12:33:24.598975 29698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:24.600113 29699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354f46db004846c1bda42c45a1135415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354f46db004846c1bda42c45a1135415" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 42235 } } }
I20260806 12:33:24.600212 29699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:24.601478 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 12:33:24.603767 29713 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:24.603892 29713 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:24.603955 29705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:24.604914 29705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:24.610677 29705 catalog_manager.cc:1347] Generated new cluster ID: 30c420bbb58f4205b916deb02531e20d
I20260806 12:33:24.610733 29705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:24.612455 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:24.603622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40381 (local address 127.6.152.190:42235)
0806 12:33:24.603723 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:24.603727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:24.604025 (+ 298us) server_negotiation.cc:407] Connection header received
0806 12:33:24.604206 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:24.604212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:24.604263 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:24.604350 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:24.605148 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:24.605938 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:24.606993 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:24.607304 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:24.607418 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:24.608800 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:24.608803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:24.608806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:24.608831 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:24.610537 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:24.611056 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:24.611060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:24.611062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:24.611170 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:24.611485 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:24.611489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:24.611490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:24.611704 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:24.611805 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:24.612188 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 12:33:24.612307 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:33:24.624243 29705 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:24.625350 29705 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:24.644176 29705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354f46db004846c1bda42c45a1135415: Generated new TSK 0
I20260806 12:33:24.644806 29705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:24.651670 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29623
2026-08-06T12:33:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:24 dist-test-slave-1xrh krb5kdc[29724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:24 dist-test-slave-1xrh krb5kdc[29724](info): set up 2 sockets
Aug 06 12:33:24 dist-test-slave-1xrh krb5kdc[29724](info): commencing operation
krb5kdc: starting...
W20260806 12:33:25.703776 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:25 dist-test-slave-1xrh krb5kdc[29724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:25.826381 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:46697
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:46697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:25.933627 29737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:25.933892 29737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:25.933974 29737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:25.937947 29737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:25.938026 29737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:25.938074 29737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:25.938115 29737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:25.938169 29737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:25.943235 29737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:46697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:46697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:25.944170 29737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:25.945101 29737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:33:25.953540 29737 server_base.cc:1034] running on GCE node
W20260806 12:33:25.953495 29742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:25.953509 29745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:25.953495 29743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:25.954294 29737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:25.955375 29737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:25.956562 29737 hybrid_clock.cc:648] HybridClock initialized: now 1786019605956544 us; error 29 us; skew 500 ppm
Aug 06 12:33:25 dist-test-slave-1xrh krb5kdc[29724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019605, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:25.959266 29737 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:25.960067 29737 webserver.cc:460] Webserver started at http://127.6.152.190:46361/ using document root <none> and password file <none>
I20260806 12:33:25.960623 29737 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:25.960714 29737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:25.960954 29737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:25.963004 29737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "7d0823733f2b49cba5284101b922e4c2"
format_stamp: "Formatted at 2026-08-06 12:33:25 on dist-test-slave-1xrh"
server_key: "4ebcef5c7f432fc53c87ce25cb30f53a"
server_key_iv: "ef0e4101928d2c1b5e02da38cbad56c1"
server_key_version: "encryptionkey@0"
I20260806 12:33:25.963608 29737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "7d0823733f2b49cba5284101b922e4c2"
format_stamp: "Formatted at 2026-08-06 12:33:25 on dist-test-slave-1xrh"
server_key: "4ebcef5c7f432fc53c87ce25cb30f53a"
server_key_iv: "ef0e4101928d2c1b5e02da38cbad56c1"
server_key_version: "encryptionkey@0"
I20260806 12:33:25.967192 29737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:33:25.969651 29752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:25.971293 29737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:33:25.971470 29737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "7d0823733f2b49cba5284101b922e4c2"
format_stamp: "Formatted at 2026-08-06 12:33:25 on dist-test-slave-1xrh"
server_key: "4ebcef5c7f432fc53c87ce25cb30f53a"
server_key_iv: "ef0e4101928d2c1b5e02da38cbad56c1"
server_key_version: "encryptionkey@0"
I20260806 12:33:25.971608 29737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:25.994215 29737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:25.997210 29737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:25.997426 29737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:26.004637 29737 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:46697
I20260806 12:33:26.004666 29806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:46697 every 8 connection(s)
I20260806 12:33:26.005568 29737 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:26.008262 29807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:26.012158 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29737
I20260806 12:33:26.012315 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:26.012621 6754 external_mini_cluster.cc:1428] Setting key 6496c576556905ef16ade40fe11adf10
I20260806 12:33:26.014083 29807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2: Bootstrap starting.
I20260806 12:33:26.016438 29807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:26.017167 29807 log.cc:824] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:26.019239 29807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2: No bootstrap required, opened a new log
I20260806 12:33:26.021589 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:26.014155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45533 (local address 127.6.152.190:46697)
0806 12:33:26.014608 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:26.014633 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:26.014669 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:33:26.015373 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:26.015390 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:26.015724 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:26.015990 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:26.016330 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:26.017380 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:26.018493 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:26.018948 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:26.019080 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:26.019931 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:26.019936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:26.019949 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:26.020359 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:26.020410 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:26.020432 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:26.020512 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:26.020670 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 12:33:26.020892 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":147,"threads_started":1}
I20260806 12:33:26.023095 29807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0823733f2b49cba5284101b922e4c2" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46697 } }
I20260806 12:33:26.023407 29807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:26.023495 29807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d0823733f2b49cba5284101b922e4c2, State: Initialized, Role: FOLLOWER
I20260806 12:33:26.023994 29807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0823733f2b49cba5284101b922e4c2" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46697 } }
I20260806 12:33:26.024180 29807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:26.024246 29807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:26.024344 29807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:26.024575 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:26.025300 29807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0823733f2b49cba5284101b922e4c2" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46697 } }
I20260806 12:33:26.025624 29807 leader_election.cc:302] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d0823733f2b49cba5284101b922e4c2; no voters:
I20260806 12:33:26.025959 29807 leader_election.cc:288] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:26.026014 29815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:26.026257 29815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [term 1 LEADER]: Becoming Leader. State: Replica: 7d0823733f2b49cba5284101b922e4c2, State: Running, Role: LEADER
I20260806 12:33:26.026613 29815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0823733f2b49cba5284101b922e4c2" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46697 } }
I20260806 12:33:26.027213 29807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:26.027837 29818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d0823733f2b49cba5284101b922e4c2. Latest consensus state: current_term: 1 leader_uuid: "7d0823733f2b49cba5284101b922e4c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0823733f2b49cba5284101b922e4c2" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46697 } } }
I20260806 12:33:26.027980 29818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:26.028332 29817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d0823733f2b49cba5284101b922e4c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0823733f2b49cba5284101b922e4c2" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 46697 } } }
I20260806 12:33:26.028434 29817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:26.028419 29824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:26.031383 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:26.026654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34489 (local address 127.6.152.190:46697)
0806 12:33:26.026770 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:26.026774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:26.027214 (+ 440us) server_negotiation.cc:407] Connection header received
0806 12:33:26.027369 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:26.027372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:26.027418 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:26.027509 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:26.027791 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:26.028540 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:26.029636 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:26.029980 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:26.030093 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:26.030906 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:26.030910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:26.030913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:26.030925 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:26.030955 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:26.030972 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:26.031047 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:26.031206 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 12:33:26.031257 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:33:26.031805 29824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:26.037241 29824 catalog_manager.cc:1347] Generated new cluster ID: c00359064dd6414c8659e4654f925be0
I20260806 12:33:26.037304 29824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:26.059458 29824 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:26.060467 29824 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:26.064800 29824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d0823733f2b49cba5284101b922e4c2: Generated new TSK 0
I20260806 12:33:26.065428 29824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:26.071828 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29737
2026-08-06T12:33:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:26 dist-test-slave-1xrh krb5kdc[29838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:26 dist-test-slave-1xrh krb5kdc[29838](info): set up 2 sockets
Aug 06 12:33:26 dist-test-slave-1xrh krb5kdc[29838](info): commencing operation
krb5kdc: starting...
W20260806 12:33:27.123724 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:27 dist-test-slave-1xrh krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:27.258528 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45841
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:45841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:27.373782 29851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:27.374051 29851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:27.374094 29851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:27.378001 29851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:27.378054 29851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:27.378073 29851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:27.378090 29851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:27.378105 29851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:27.383318 29851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:45841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:27.384325 29851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:27.385296 29851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:27.393613 29859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:27.393607 29857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:27.393607 29856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:27.393702 29851 server_base.cc:1034] running on GCE node
I20260806 12:33:27.394485 29851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:27.395612 29851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:27.396783 29851 hybrid_clock.cc:648] HybridClock initialized: now 1786019607396767 us; error 28 us; skew 500 ppm
Aug 06 12:33:27 dist-test-slave-1xrh krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019607, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:27.399537 29851 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:27.400370 29851 webserver.cc:460] Webserver started at http://127.6.152.190:36103/ using document root <none> and password file <none>
I20260806 12:33:27.400872 29851 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:27.400938 29851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:27.401100 29851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:27.403388 29851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "41e088ef34794605950d6d49a2838745"
format_stamp: "Formatted at 2026-08-06 12:33:27 on dist-test-slave-1xrh"
server_key: "acb8747c285ad37db1af39710c208775"
server_key_iv: "d88c4ecb8b0987098ee2c755ccdd8fbc"
server_key_version: "encryptionkey@0"
I20260806 12:33:27.403949 29851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "41e088ef34794605950d6d49a2838745"
format_stamp: "Formatted at 2026-08-06 12:33:27 on dist-test-slave-1xrh"
server_key: "acb8747c285ad37db1af39710c208775"
server_key_iv: "d88c4ecb8b0987098ee2c755ccdd8fbc"
server_key_version: "encryptionkey@0"
I20260806 12:33:27.407660 29851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 12:33:27.410007 29866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:27.411288 29851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:27.411386 29851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "41e088ef34794605950d6d49a2838745"
format_stamp: "Formatted at 2026-08-06 12:33:27 on dist-test-slave-1xrh"
server_key: "acb8747c285ad37db1af39710c208775"
server_key_iv: "d88c4ecb8b0987098ee2c755ccdd8fbc"
server_key_version: "encryptionkey@0"
I20260806 12:33:27.411468 29851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:27.415853 29851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:27.419088 29851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:27.419315 29851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:27.426093 29851 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:45841
I20260806 12:33:27.426107 29920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:45841 every 8 connection(s)
I20260806 12:33:27.426971 29851 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:27.429656 29921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:27.430073 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29851
I20260806 12:33:27.430262 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:27.430565 6754 external_mini_cluster.cc:1428] Setting key 86925e560270f9579b85135b260aad5f
I20260806 12:33:27.435914 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:27.431963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49257 (local address 127.6.152.190:45841)
0806 12:33:27.432410 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:27.432426 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:27.432462 (+ 36us) server_negotiation.cc:407] Connection header received
0806 12:33:27.433190 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:27.433207 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:27.433511 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:27.433817 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:27.434669 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:27.434674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:27.434686 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:27.434697 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:27.434757 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:27.434776 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:27.434893 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:27.435038 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 12:33:27.435253 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":134,"threads_started":1}
I20260806 12:33:27.436002 29921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745: Bootstrap starting.
I20260806 12:33:27.438484 29921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:27.438848 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:27.439364 29921 log.cc:824] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:27.441447 29921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745: No bootstrap required, opened a new log
I20260806 12:33:27.442660 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:27.440986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60871 (local address 127.6.152.190:45841)
0806 12:33:27.441096 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:27.441100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:27.441288 (+ 188us) server_negotiation.cc:407] Connection header received
0806 12:33:27.441430 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:27.441435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:27.441473 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:27.441559 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:27.442266 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:27.442269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:27.442272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 12:33:27.442283 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:27.442317 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 12:33:27.442328 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:27.442422 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:27.442500 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 12:33:27.442539 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:33:27.445145 29921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e088ef34794605950d6d49a2838745" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45841 } }
I20260806 12:33:27.445343 29921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 12:33:27.445375 29921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e088ef34794605950d6d49a2838745, State: Initialized, Role: FOLLOWER
I20260806 12:33:27.445801 29921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e088ef34794605950d6d49a2838745" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45841 } }
I20260806 12:33:27.445942 29921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 12:33:27.446034 29921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 12:33:27.446152 29921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:27.446988 29921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e088ef34794605950d6d49a2838745" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45841 } }
I20260806 12:33:27.447330 29921 leader_election.cc:302] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41e088ef34794605950d6d49a2838745; no voters:
I20260806 12:33:27.447643 29921 leader_election.cc:288] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 12:33:27.447751 29931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:27.448033 29931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [term 1 LEADER]: Becoming Leader. State: Replica: 41e088ef34794605950d6d49a2838745, State: Running, Role: LEADER
I20260806 12:33:27.448340 29931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e088ef34794605950d6d49a2838745" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45841 } }
I20260806 12:33:27.448783 29921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:27.449852 29933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41e088ef34794605950d6d49a2838745. Latest consensus state: current_term: 1 leader_uuid: "41e088ef34794605950d6d49a2838745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e088ef34794605950d6d49a2838745" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45841 } } }
I20260806 12:33:27.449982 29933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:27.450160 29932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41e088ef34794605950d6d49a2838745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e088ef34794605950d6d49a2838745" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 45841 } } }
I20260806 12:33:27.450307 29932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:27.450632 29940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:27.453462 29940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:27.459059 29940 catalog_manager.cc:1347] Generated new cluster ID: 70e892859e4443b6843410084eb8dcd9
I20260806 12:33:27.459147 29940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:27.471071 29940 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:27.471408 29940 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:27.479406 29940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41e088ef34794605950d6d49a2838745: Generated new TSK 0
I20260806 12:33:27.480155 29940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 12:33:27.486996 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29851
2026-08-06T12:33:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1414 ms)
[----------] 18 tests from AuthTokenIssuingTest (25789 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:27 dist-test-slave-1xrh krb5kdc[29952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:27 dist-test-slave-1xrh krb5kdc[29952](info): set up 2 sockets
Aug 06 12:33:27 dist-test-slave-1xrh krb5kdc[29952](info): commencing operation
krb5kdc: starting...
W20260806 12:33:28.539623 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:28 dist-test-slave-1xrh krb5kdc[29952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:28.666518 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:34291
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:34291,127.6.152.189:41481,127.6.152.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:28.776540 29965 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 12:33:28.776813 29965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:28.776836 29965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:28.776876 29965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:28.780798 29965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:28.780845 29965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:28.780864 29965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:28.780880 29965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:28.780895 29965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:28.785916 29965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:34291,127.6.152.189:41481,127.6.152.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:34291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:28.786986 29965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:28.788000 29965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:28.796325 29971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:28.796348 29970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:28.796446 29965 server_base.cc:1034] running on GCE node
W20260806 12:33:28.796329 29973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:28.797149 29965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:28.798127 29965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:28.799278 29965 hybrid_clock.cc:648] HybridClock initialized: now 1786019608799250 us; error 37 us; skew 500 ppm
Aug 06 12:33:28 dist-test-slave-1xrh krb5kdc[29952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:28.809484 29965 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:28.810334 29965 webserver.cc:460] Webserver started at http://127.6.152.190:34383/ using document root <none> and password file <none>
I20260806 12:33:28.810921 29965 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:28.811015 29965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:28.811296 29965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:28.813257 29965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "1defb01d31c441c8873ff2ffd0de2263"
format_stamp: "Formatted at 2026-08-06 12:33:28 on dist-test-slave-1xrh"
server_key: "33951cae362e34aa510b75abb7882721"
server_key_iv: "5f16e3f37a46597d06010362654b312c"
server_key_version: "encryptionkey@0"
I20260806 12:33:28.813848 29965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "1defb01d31c441c8873ff2ffd0de2263"
format_stamp: "Formatted at 2026-08-06 12:33:28 on dist-test-slave-1xrh"
server_key: "33951cae362e34aa510b75abb7882721"
server_key_iv: "5f16e3f37a46597d06010362654b312c"
server_key_version: "encryptionkey@0"
I20260806 12:33:28.817503 29965 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 12:33:28.819742 29980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:28.820899 29965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:28.821033 29965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "1defb01d31c441c8873ff2ffd0de2263"
format_stamp: "Formatted at 2026-08-06 12:33:28 on dist-test-slave-1xrh"
server_key: "33951cae362e34aa510b75abb7882721"
server_key_iv: "5f16e3f37a46597d06010362654b312c"
server_key_version: "encryptionkey@0"
I20260806 12:33:28.821138 29965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:28.840955 29965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:28.844092 29965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:28.844290 29965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:28.851385 29965 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:34291
I20260806 12:33:28.851686 30032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:34291 every 8 connection(s)
I20260806 12:33:28.852319 29965 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:28.853005 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 29965
I20260806 12:33:28.853144 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:28.853438 6754 external_mini_cluster.cc:1428] Setting key 19bf36841c041e807b215f819da20d0b
I20260806 12:33:28.855166 30033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:28.860706 30033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.152.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.189@KRBTEST.COM" created.
Aug 06 12:33:28 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:28.877527 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:28.863405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.6.152.190:34291)
0806 12:33:28.863850 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:28.863866 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:28.864066 (+ 200us) server_negotiation.cc:407] Connection header received
0806 12:33:28.865114 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:28.865123 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:28.865343 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:28.865503 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:28.866492 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:28.867869 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:28.869228 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:28.869898 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:28.870090 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:28.872734 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:28.872739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:28.872748 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:28.872767 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:28.874375 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:28.874836 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:28.874848 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:28.874849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:28.874991 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:28.875348 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:28.875353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:28.875355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:28.875703 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:28.875852 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:28.876239 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 12:33:28.876509 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"spinlock_wait_cycles":33664,"thread_start_us":159,"threads_started":1}
I20260806 12:33:28.877528 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:28.863405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:34291 (local address 127.0.0.1:49808)
0806 12:33:28.863894 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:28.863942 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:28.864770 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:28.865542 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:28.865546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:28.865953 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:28.866315 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:28.866331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:28.868017 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:28.868023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:28.869106 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:28.869116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:28.870101 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:28.870105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:28.870214 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:28.870842 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:28.870868 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:28.872583 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:28.874487 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:28.874498 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:28.874507 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:28.874724 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:28.875088 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:28.875091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:28.875093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:28.875256 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:28.875864 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:28.875872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:28.876005 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 12:33:28.876167 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 12:33:28.876584 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":105,"thread_start_us":129,"threads_started":1}
I20260806 12:33:28.880075 30033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:28.880812 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:28.880549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50326)
0806 12:33:28.880664 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:28.880731 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 12:33:28.881055 29984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab.
Entry for principal kudu/127.6.152.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab.
I20260806 12:33:28.881983 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:28.881719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50340)
0806 12:33:28.881873 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:28.881931 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 12:33:28.882143 30033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:41481: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
I20260806 12:33:28.883060 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:41481
--webserver_interface=127.6.152.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab
--principal=kudu/127.6.152.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:34291,127.6.152.189:41481,127.6.152.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:33:28.931403 30033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } attempt: 1
I20260806 12:33:28.932255 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:28.931951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50344)
0806 12:33:28.932108 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:28.932177 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 12:33:28.933022 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:28.932789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50352)
0806 12:33:28.932902 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:28.932964 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 12:33:28.933166 30033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:41481: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
W20260806 12:33:28.997103 30040 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 12:33:28.997345 30040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:28.997366 30040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:28.997400 30040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:29.001362 30040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:29.001416 30040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:29.001435 30040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:29.001451 30040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260806 12:33:29.001339 30033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } attempt: 2
W20260806 12:33:29.001467 30040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:29.002207 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.001875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50354)
0806 12:33:29.002042 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.002112 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 12:33:29.002965 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.002741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50368)
0806 12:33:29.002843 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.002891 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 12:33:29.003099 30033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:41481: Network error: Client connection negotiation failed: client connection to 127.6.152.189:41481: connect: Connection refused (error 111)
I20260806 12:33:29.006783 30040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:34291,127.6.152.189:41481,127.6.152.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab
--principal=kudu/127.6.152.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:41481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.152.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:29.007903 30040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:29.008948 30040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:33:29.017429 30040 server_base.cc:1034] running on GCE node
W20260806 12:33:29.017380 30045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:29.017401 30046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:29.017361 30048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:29.018213 30040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:29.019255 30040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:29.020419 30040 hybrid_clock.cc:648] HybridClock initialized: now 1786019609020407 us; error 23 us; skew 500 ppm
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:29.023316 30040 init.cc:377] Logged in from keytab as kudu/127.6.152.189@KRBTEST.COM (short username kudu)
I20260806 12:33:29.024214 30040 webserver.cc:460] Webserver started at http://127.6.152.189:40099/ using document root <none> and password file <none>
I20260806 12:33:29.024827 30040 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:29.024914 30040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:29.025107 30040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:29.027067 30040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data/instance:
uuid: "72441edc2da14e85b75035660ec80953"
format_stamp: "Formatted at 2026-08-06 12:33:29 on dist-test-slave-1xrh"
server_key: "d62771600dd4d76a1e261facc7f8a75b"
server_key_iv: "b19d784d897dad9a9d28f849a7b01dbf"
server_key_version: "encryptionkey@0"
I20260806 12:33:29.027640 30040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal/instance:
uuid: "72441edc2da14e85b75035660ec80953"
format_stamp: "Formatted at 2026-08-06 12:33:29 on dist-test-slave-1xrh"
server_key: "d62771600dd4d76a1e261facc7f8a75b"
server_key_iv: "b19d784d897dad9a9d28f849a7b01dbf"
server_key_version: "encryptionkey@0"
I20260806 12:33:29.031081 30040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 12:33:29.033321 30057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:29.034466 30040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:29.034610 30040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal
uuid: "72441edc2da14e85b75035660ec80953"
format_stamp: "Formatted at 2026-08-06 12:33:29 on dist-test-slave-1xrh"
server_key: "d62771600dd4d76a1e261facc7f8a75b"
server_key_iv: "b19d784d897dad9a9d28f849a7b01dbf"
server_key_version: "encryptionkey@0"
I20260806 12:33:29.034705 30040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:29.045322 30040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:29.048414 30040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:29.048614 30040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:29.055234 30040 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.189:41481
I20260806 12:33:29.055507 30109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.189:41481 every 8 connection(s)
I20260806 12:33:29.056150 30040 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
I20260806 12:33:29.058741 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:29.059758 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 30040
I20260806 12:33:29.059886 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-1/wal/instance
I20260806 12:33:29.060146 6754 external_mini_cluster.cc:1428] Setting key fc0d5b4a27fefd40340c3586edd28d71
I20260806 12:33:29.062789 30110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.152.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.188@KRBTEST.COM" created.
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:29.077560 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.065032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.6.152.190:34291)
0806 12:33:29.065213 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.065219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.065945 (+ 726us) server_negotiation.cc:407] Connection header received
0806 12:33:29.066835 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.066842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.066906 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.067010 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.068108 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.069014 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.070239 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.070472 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.070554 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.073077 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.073083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.073088 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.073123 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.075225 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.075865 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.075868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.075869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.075939 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.076404 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.076406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.076407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.076584 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.076656 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.077333 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.077432 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:33:29.078676 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.065348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:34291 (local address 127.0.0.1:49810)
0806 12:33:29.065778 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.065828 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.066632 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.067201 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.067208 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.067529 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.067955 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.067967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.069144 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.069147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.070120 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.070130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.070600 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.070603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.070741 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.071375 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.071391 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.072908 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.075339 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.075348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.075365 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.075762 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.076057 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.076060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.076062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.076269 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.076702 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.076711 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.077052 (+ 341us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.077290 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.077656 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
I20260806 12:33:29.080763 30110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab.
Entry for principal kudu/127.6.152.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab.
I20260806 12:33:29.090670 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.081281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50382)
0806 12:33:29.081473 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.081487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.081569 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.082242 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.082299 (+ 57us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.082493 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.082596 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.082602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.083850 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.083852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.084427 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.084436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.085104 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.085107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.085182 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.085749 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.085763 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.086922 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.089115 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.089117 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.089119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.089306 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.089600 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.089602 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.089603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.089680 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.090188 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.090194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.090297 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.090384 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.090500 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260806 12:33:29.090869 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.081416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.6.152.189:41481)
0806 12:33:29.081810 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.081823 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.081846 (+ 23us) server_negotiation.cc:407] Connection header received
0806 12:33:29.081886 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.081891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.082088 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.082226 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.082725 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.083732 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.084537 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.084965 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.085084 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.087061 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.087071 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.087085 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.087109 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.088985 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.089404 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.089408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.089409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.089509 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.089759 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.089762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.089763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.090067 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.090206 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.090515 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.090630 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":162,"threads_started":1}
I20260806 12:33:29.091317 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:40785
--webserver_interface=127.6.152.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab
--principal=kudu/127.6.152.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:34291,127.6.152.189:41481,127.6.152.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:33:29.092828 30110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:29.093557 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.093315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39312)
0806 12:33:29.093429 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.093502 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 12:33:29.093758 30059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
I20260806 12:33:29.094285 30033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } attempt: 3
I20260806 12:33:29.094717 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.094411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39316)
0806 12:33:29.094582 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.094638 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 12:33:29.094898 30110 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:40785: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:33:29.102541 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.094758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.6.152.189:41481)
0806 12:33:29.094854 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.094858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.094989 (+ 131us) server_negotiation.cc:407] Connection header received
0806 12:33:29.095157 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.095160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.095205 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.095307 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.095726 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.096261 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.097025 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.097288 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.097393 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.099545 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.099548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.099551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.099574 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.100982 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.101396 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.101400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.101402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.101490 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.101763 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.101768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.101769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.101938 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.102040 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.102295 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.102397 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 12:33:29.102541 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.094724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50392)
0806 12:33:29.094900 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.094913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.095033 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.095325 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.095328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.095518 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.095615 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.095620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.096367 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.096371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.096902 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.096909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.097397 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.097401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.097515 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.097978 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.097990 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.099412 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.101097 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.101100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.101102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.101298 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.101582 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.101586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.101587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.101674 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.102033 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.102036 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.102137 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.102244 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.102376 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 12:33:29.103549 30033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:29.104255 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.104004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39320)
0806 12:33:29.104128 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.104185 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 12:33:29.104959 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.104707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39322)
0806 12:33:29.104852 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.104901 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260806 12:33:29.105087 30033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:40785: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
I20260806 12:33:29.136241 30033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } attempt: 1
I20260806 12:33:29.137001 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.136698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39328)
0806 12:33:29.136848 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.136915 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 12:33:29.137720 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.137499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39338)
0806 12:33:29.137605 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.137657 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 12:33:29.137913 30033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:40785: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
I20260806 12:33:29.144140 30110 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } attempt: 1
I20260806 12:33:29.144851 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.144528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39354)
0806 12:33:29.144683 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.144758 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 12:33:29.145563 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.145338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39370)
0806 12:33:29.145445 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.145501 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 12:33:29.145697 30110 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:40785: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
W20260806 12:33:29.204192 30116 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 12:33:29.204450 30116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:29.204475 30116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:29.204512 30116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:29.208706 30116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:29.208760 30116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:29.208792 30116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:29.208833 30116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:29.208849 30116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:29.213073 30033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } attempt: 2
I20260806 12:33:29.213857 30110 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } attempt: 2
I20260806 12:33:29.214007 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.213602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39386)
0806 12:33:29.213802 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.213882 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 12:33:29.214546 30116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:34291,127.6.152.189:41481,127.6.152.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab
--principal=kudu/127.6.152.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:40785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.152.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:29.214783 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.214523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39406)
0806 12:33:29.214625 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.214698 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 12:33:29.214900 30033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:40785: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
I20260806 12:33:29.214821 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.214293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39398)
0806 12:33:29.214686 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.214745 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 12:33:29.215613 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.215371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39422)
0806 12:33:29.215501 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.215553 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 12:33:29.215765 30110 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:40785: Network error: Client connection negotiation failed: client connection to 127.6.152.188:40785: connect: Connection refused (error 111)
I20260806 12:33:29.215865 30116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:29.216853 30116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:29.225118 30122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:29.225174 30125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:29.225129 30123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:29.225422 30116 server_base.cc:1034] running on GCE node
I20260806 12:33:29.225930 30116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:29.227020 30116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:29.228217 30116 hybrid_clock.cc:648] HybridClock initialized: now 1786019609228171 us; error 63 us; skew 500 ppm
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:29.231019 30116 init.cc:377] Logged in from keytab as kudu/127.6.152.188@KRBTEST.COM (short username kudu)
I20260806 12:33:29.231933 30116 webserver.cc:460] Webserver started at http://127.6.152.188:33781/ using document root <none> and password file <none>
I20260806 12:33:29.232403 30116 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:29.232460 30116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:29.232609 30116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:29.234498 30116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data/instance:
uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7"
format_stamp: "Formatted at 2026-08-06 12:33:29 on dist-test-slave-1xrh"
server_key: "4c01bacd839b11241dfadbeb7c47e1c4"
server_key_iv: "b141b215b22a684061eb31e2a3ee713e"
server_key_version: "encryptionkey@0"
I20260806 12:33:29.234997 30116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal/instance:
uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7"
format_stamp: "Formatted at 2026-08-06 12:33:29 on dist-test-slave-1xrh"
server_key: "4c01bacd839b11241dfadbeb7c47e1c4"
server_key_iv: "b141b215b22a684061eb31e2a3ee713e"
server_key_version: "encryptionkey@0"
I20260806 12:33:29.238583 30116 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 12:33:29.240890 30132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:29.242023 30116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:29.242159 30116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal
uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7"
format_stamp: "Formatted at 2026-08-06 12:33:29 on dist-test-slave-1xrh"
server_key: "4c01bacd839b11241dfadbeb7c47e1c4"
server_key_iv: "b141b215b22a684061eb31e2a3ee713e"
server_key_version: "encryptionkey@0"
I20260806 12:33:29.242262 30116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:29.255872 30116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:29.258406 30116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:29.258618 30116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:29.265486 30116 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.188:40785
I20260806 12:33:29.265774 30184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.188:40785 every 8 connection(s)
I20260806 12:33:29.266402 30116 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
I20260806 12:33:29.268550 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 30116
I20260806 12:33:29.268674 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786019324359312-6754-0/minicluster-data/master-2/wal/instance
I20260806 12:33:29.268993 6754 external_mini_cluster.cc:1428] Setting key 662b90e7a9b13b0e37d0f1c1566dcbee
I20260806 12:33:29.269320 30185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:29.274236 30185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:29.283972 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.270323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.6.152.188:40785)
0806 12:33:29.270747 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.270767 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.270807 (+ 40us) server_negotiation.cc:407] Connection header received
0806 12:33:29.271595 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.271618 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.272003 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.272341 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.273060 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.274544 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.275486 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.276013 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.276296 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.279035 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.279041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.279052 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.279086 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.280863 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.281358 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.281363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.281367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.281472 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.281775 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.281778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.281780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.282269 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.282495 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.282751 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.283036 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":122,"threads_started":1}
I20260806 12:33:29.285200 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.276066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.6.152.190:34291)
0806 12:33:29.276193 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.276199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.276482 (+ 283us) server_negotiation.cc:407] Connection header received
0806 12:33:29.276706 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.276710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.276758 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.276826 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.277518 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.278160 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.279199 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.279404 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.279531 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.281708 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.281711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.281713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.281759 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.283432 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.283894 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.283898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.283899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.283996 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.284346 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.284349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.284351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.284587 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.284687 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.284984 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.285091 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:33:29.285269 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.275995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:34291 (local address 127.0.0.1:49818)
0806 12:33:29.276354 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.276400 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.276572 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.276891 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.276895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.277225 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.277357 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.277363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.278291 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.278295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.279034 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.279042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.279537 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.279541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.279663 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.280072 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.280083 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.281571 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.283545 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.283555 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.283558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.283782 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.284108 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.284113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.284116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.284220 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.284690 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.284695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.284786 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.284893 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.285078 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":135,"threads_started":1}
I20260806 12:33:29.286329 30185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:29.294777 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.286816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:41481 (local address 127.0.0.1:50402)
0806 12:33:29.287175 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.287188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.287258 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.287575 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.287578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.287737 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.287816 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.287821 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.288569 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.288574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.289059 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.289067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.289502 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.289506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.289592 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.289989 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.290000 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.291558 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.293196 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.293200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.293204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.293440 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.293718 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.293720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.293721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.293782 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.294262 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.294268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.294363 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.294483 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.294612 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260806 12:33:29.294873 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.287024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.6.152.189:41481)
0806 12:33:29.287207 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.287212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.287230 (+ 18us) server_negotiation.cc:407] Connection header received
0806 12:33:29.287385 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.287389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.287455 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.287560 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.287981 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.288452 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.289175 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.289387 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.289496 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.291701 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.291704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.291706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.291730 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.293110 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.293544 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.293547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.293549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.293629 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.293885 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.293890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.293893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.294162 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.294253 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.294584 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.294712 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:33:29.295575 30185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:29.295886 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.286377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49828 (local address 127.6.152.190:34291)
0806 12:33:29.286506 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.286522 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.286536 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:33:29.286674 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.286680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.286730 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.286828 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.287579 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.288147 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.288857 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.289075 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.289173 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.292383 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.292387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.292389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.292443 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.294065 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.294521 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.294526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.294527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.294610 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.294936 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.294942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.294946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.295220 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.295485 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.295626 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.295739 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:33:29.304653 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.296036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39442)
0806 12:33:29.296318 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.296332 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.296417 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.296634 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.296636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.296755 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.296874 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.296885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.297738 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.297742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.298365 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.298374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.299647 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.299650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.299765 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.300129 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.300140 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.301463 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.303084 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.303090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.303095 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.303354 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.303640 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.303642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.303643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.303705 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.304145 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.304148 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.304253 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.304373 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.304511 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":16}
I20260806 12:33:29.304809 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.296084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.6.152.188:40785)
0806 12:33:29.296218 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.296221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.296376 (+ 155us) server_negotiation.cc:407] Connection header received
0806 12:33:29.296503 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.296506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.296540 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.296615 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.297010 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.297617 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.298482 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.299522 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.299638 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.301622 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.301625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.301627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.301647 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.302997 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.303464 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.303468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.303470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.303553 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.303828 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.303831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.303833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.304048 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.304145 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.304457 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.304574 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":69}
I20260806 12:33:29.305902 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.297080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.6.152.189:41481)
0806 12:33:29.297189 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.297193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.297219 (+ 26us) server_negotiation.cc:407] Connection header received
0806 12:33:29.297257 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.297261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.297296 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.297385 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.298063 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.298560 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.299319 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.299666 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.299776 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.302289 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.302293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.302295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.302323 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.304002 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.304463 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.304467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.304469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.304553 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.305044 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.305048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.305049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.305244 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.305412 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.305641 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.305749 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 12:33:29.306901 30110 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } attempt: 3
I20260806 12:33:29.308182 30185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: Bootstrap starting.
I20260806 12:33:29.309414 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:29.311275 30185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:29.312234 30185 log.cc:824] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: Log is configured to *not* fsync() on all Append() calls
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019609, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:33:29.314958 30185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: No bootstrap required, opened a new log
I20260806 12:33:29.315207 30033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } attempt: 3
I20260806 12:33:29.317198 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.307300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39446)
0806 12:33:29.307554 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.307569 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.307642 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.308068 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.308071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.308245 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.308341 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.308348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.309188 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.309191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.310232 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.310243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.311008 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.311012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.311163 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.311657 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.311670 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.313366 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.315541 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.315546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.315551 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.315817 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.316167 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.316172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.316174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.316269 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.316745 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.316748 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.316845 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.316949 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.317078 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260806 12:33:29.317667 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.307770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.6.152.188:40785)
0806 12:33:29.307866 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.307869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.307881 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:33:29.307912 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.307914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.307949 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.308038 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.308474 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.309059 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.310433 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.310847 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.310962 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.313539 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.313547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.313564 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.313593 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.315396 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.315956 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.315959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.315961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.316049 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.316401 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.316407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.316409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.316617 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.317389 (+ 772us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.317462 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.317570 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 12:33:29.314441 30007 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1defb01d31c441c8873ff2ffd0de2263"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49828
I20260806 12:33:29.319456 30185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.319696 30185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2f72def9ce47bfbf2b0b4ca63150f7, State: Initialized, Role: FOLLOWER
I20260806 12:33:29.320030 30185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.320575 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.320776 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.320924 30110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: Bootstrap starting.
I20260806 12:33:29.321441 30185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 12:33:29 dist-test-slave-1xrh krb5kdc[29952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019608, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:33:29.325584 30110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: Neither blocks nor log segments found. Creating new log.
W20260806 12:33:29.326438 30203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:29.326530 30203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:29.326640 30110 log.cc:824] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:29.329077 30110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: No bootstrap required, opened a new log
I20260806 12:33:29.325763 30116 master_runner.cc:186] Error getting master registration for 127.6.152.190:34291: OK, instance_id { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" instance_seqno: 1786019608849424 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:33:29.329778 30116 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:33:29.330641 30036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.315812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:40785 (local address 127.0.0.1:39448)
0806 12:33:29.315983 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:29.316128 (+ 145us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:29.316230 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:29.318047 (+ 1817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:29.318050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:29.318249 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:29.318357 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.318364 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.319251 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.319258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.320029 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:29.320042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.320646 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.320650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:29.320734 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.321246 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:29.321261 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:29.324275 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:29.328134 (+ 3859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.328137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.328140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.328442 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.328976 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:29.328979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:29.328982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:29.329086 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:29.330098 (+ 1012us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:29.330102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:29.330207 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 12:33:29.330308 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 12:33:29.330466 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 12:33:29.331883 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.316594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.6.152.188:40785)
0806 12:33:29.317838 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.317842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.317854 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:33:29.317884 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.317886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.317924 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.318012 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.318449 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.319115 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.320204 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.320500 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.321187 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.324475 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.324482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.324487 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.324544 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.327940 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.328638 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.328642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.328643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.328817 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.329355 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.329362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.329391 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.329911 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.331215 (+ 1304us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.331397 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.331607 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":102,"threads_started":1}
I20260806 12:33:29.333698 30110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.333940 30110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72441edc2da14e85b75035660ec80953, State: Initialized, Role: FOLLOWER
I20260806 12:33:29.334283 30110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.334830 30206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.335100 30206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.335191 30033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263: Bootstrap starting.
I20260806 12:33:29.335728 30110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:29.337903 30033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:29.338739 30033 log.cc:824] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:29.340700 30033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263: No bootstrap required, opened a new log
W20260806 12:33:29.343400 30217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:29.343559 30217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:29.343495 30033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.339567 30040 master_runner.cc:186] Error getting master registration for 127.6.152.190:34291: OK, instance_id { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" instance_seqno: 1786019608849424 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:33:29.343688 30033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1defb01d31c441c8873ff2ffd0de2263, State: Initialized, Role: FOLLOWER
I20260806 12:33:29.343875 30040 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:33:29.344033 30033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.345209 30033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:29.345332 30007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 12:33:29.345474 30007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 0 FOLLOWER]: Advancing to term 1
W20260806 12:33:29.345728 29986 tablet.cc:2369] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:33:29.346899 30007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.348032 30007 leader_election.cc:288] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [CANDIDATE]: Term 1 election: Requested vote from peers 72441edc2da14e85b75035660ec80953 (127.6.152.189:41481), 0a2f72def9ce47bfbf2b0b4ca63150f7 (127.6.152.188:40785)
I20260806 12:33:29.348630 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1defb01d31c441c8873ff2ffd0de2263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7"
I20260806 12:33:29.348692 30084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1defb01d31c441c8873ff2ffd0de2263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72441edc2da14e85b75035660ec80953"
I20260806 12:33:29.348863 30159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:29.348938 30084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:29.350219 30159 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1defb01d31c441c8873ff2ffd0de2263 in term 1.
I20260806 12:33:29.350224 30084 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1defb01d31c441c8873ff2ffd0de2263 in term 1.
I20260806 12:33:29.350663 29984 leader_election.cc:302] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1defb01d31c441c8873ff2ffd0de2263, 72441edc2da14e85b75035660ec80953; no voters:
I20260806 12:33:29.351574 30238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.351711 30238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.351822 30238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:29.351909 30238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [term 1 LEADER]: Becoming Leader. State: Replica: 1defb01d31c441c8873ff2ffd0de2263, State: Running, Role: LEADER
I20260806 12:33:29.352203 30238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } }
I20260806 12:33:29.355908 30220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1defb01d31c441c8873ff2ffd0de2263. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.356027 30220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:29.356901 30241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:29.360004 30241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:29.366637 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.350596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.6.152.190:34291)
0806 12:33:29.350721 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.350726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.355878 (+ 5152us) server_negotiation.cc:407] Connection header received
0806 12:33:29.356148 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.356152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.356217 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.356276 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.357512 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.358302 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.359696 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.360079 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.360164 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.362481 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.362486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.362489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.362517 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.364512 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.365105 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.365109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.365111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.365209 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.365574 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.365578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.365579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.365817 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.365933 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.366338 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.366467 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 12:33:29.375065 30188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.355750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.6.152.188:40785)
0806 12:33:29.355869 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.355874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.363535 (+ 7661us) server_negotiation.cc:407] Connection header received
0806 12:33:29.363693 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.363697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.363745 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.363808 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.366487 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.367194 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.368674 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.369069 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.369180 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.371059 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.371064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.371067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.371097 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.373000 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.373535 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.373539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.373541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.373627 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.373962 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.373966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.373968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.374227 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.374330 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.374742 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.374885 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 12:33:29.377259 30159 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [term 1 FOLLOWER]: Refusing update from remote peer 1defb01d31c441c8873ff2ffd0de2263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:29.377607 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:29.351033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.6.152.189:41481)
0806 12:33:29.351172 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.351177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.362249 (+ 11072us) server_negotiation.cc:407] Connection header received
0806 12:33:29.362423 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.362426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.362471 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.362523 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.364625 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.365365 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.368701 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:29.369046 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:29.369119 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:29.371951 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:29.371955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:29.371957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:29.372004 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:29.373669 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.376224 (+ 2555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.376229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.376231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.376328 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:29.376644 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:29.376648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:29.376649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:29.376875 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:29.376987 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:29.377326 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 12:33:29.377449 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 12:33:29.377902 30238 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:29.379432 30084 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [term 1 FOLLOWER]: Refusing update from remote peer 1defb01d31c441c8873ff2ffd0de2263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:29.382300 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1defb01d31c441c8873ff2ffd0de2263. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.382431 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.382966 30238 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:29.385977 30220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.386092 30220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:29.386329 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.386448 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.387313 30243 mvcc.cc:205] Tried to move back new op lower bound from 7315536320001179648 to 7315536319909470208. Current Snapshot: MvccSnapshot[applied={T|T < 7315536320001179648}]
I20260806 12:33:29.387293 30206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1defb01d31c441c8873ff2ffd0de2263. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.387436 30206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.388574 30238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.388684 30238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:29.389477 30245 mvcc.cc:205] Tried to move back new op lower bound from 7315536320001179648 to 7315536319909470208. Current Snapshot: MvccSnapshot[applied={T|T < 7315536320001179648}]
I20260806 12:33:29.389696 30206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1defb01d31c441c8873ff2ffd0de2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defb01d31c441c8873ff2ffd0de2263" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 34291 } } peers { permanent_uuid: "72441edc2da14e85b75035660ec80953" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 41481 } } peers { permanent_uuid: "0a2f72def9ce47bfbf2b0b4ca63150f7" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 40785 } } }
I20260806 12:33:29.389806 30206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:29.392637 30241 catalog_manager.cc:1347] Generated new cluster ID: d5ce688941db424eadb705fe81d0b8ff
I20260806 12:33:29.392697 30241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:29.393440 30247 mvcc.cc:205] Tried to move back new op lower bound from 7315536320001179648 to 7315536319909470208. Current Snapshot: MvccSnapshot[applied={T|T < 7315536320001179648}]
I20260806 12:33:29.447880 30241 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:29.449395 30241 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:29.466674 30241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1defb01d31c441c8873ff2ffd0de2263: Generated new TSK 0
I20260806 12:33:29.467602 30241 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:33:29.499006 30037 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.495432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.6.152.190:34291)
0806 12:33:29.495569 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.495574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.497632 (+ 2058us) server_negotiation.cc:407] Connection header received
0806 12:33:29.498078 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.498082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.498139 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.498197 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.498864 (+ 667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41108: BlockingRecv error: recv got EOF from 127.0.0.1:41108 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 12:33:29.500227 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.496189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.6.152.189:41481)
0806 12:33:29.496337 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.496342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.497987 (+ 1645us) server_negotiation.cc:407] Connection header received
0806 12:33:29.499417 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.499421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.499472 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.499525 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.500115 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47756: BlockingRecv error: recv got EOF from 127.0.0.1:47756 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 12:33:29.501940 30188 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.496968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.6.152.188:40785)
0806 12:33:29.497089 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.497094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.500782 (+ 3688us) server_negotiation.cc:407] Connection header received
0806 12:33:29.501169 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.501174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.501232 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.501288 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.501819 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49346: BlockingRecv error: recv got EOF from 127.0.0.1:49346 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 12:33:29.506805 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.505241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.6.152.189:41481)
0806 12:33:29.505351 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.505356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.505750 (+ 394us) server_negotiation.cc:407] Connection header received
0806 12:33:29.505978 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.505982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.506056 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.506114 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.506706 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47766: BlockingRecv error: recv got EOF from 127.0.0.1:47766 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 12:33:29.511826 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.510687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.6.152.189:41481)
0806 12:33:29.510806 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.510810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.510839 (+ 29us) server_negotiation.cc:407] Connection header received
0806 12:33:29.510980 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.510987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.511040 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.511096 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.511730 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47774: BlockingRecv error: recv got EOF from 127.0.0.1:47774 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 12:33:29.517498 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.515852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.6.152.189:41481)
0806 12:33:29.515954 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.515958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.516527 (+ 569us) server_negotiation.cc:407] Connection header received
0806 12:33:29.516726 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.516730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.516781 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.516836 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.517397 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47782: BlockingRecv error: recv got EOF from 127.0.0.1:47782 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 12:33:29.525889 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.523985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.6.152.189:41481)
0806 12:33:29.524134 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.524139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.524844 (+ 705us) server_negotiation.cc:407] Connection header received
0806 12:33:29.525053 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.525057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.525112 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.525168 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.525782 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47784: BlockingRecv error: recv got EOF from 127.0.0.1:47784 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 12:33:29.538733 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.536595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.6.152.189:41481)
0806 12:33:29.536722 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.536727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.537445 (+ 718us) server_negotiation.cc:407] Connection header received
0806 12:33:29.537688 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.537715 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.537785 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.537860 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.538612 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47786: BlockingRecv error: recv got EOF from 127.0.0.1:47786 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 12:33:29.559106 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.557231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.6.152.189:41481)
0806 12:33:29.557347 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.557352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.557985 (+ 633us) server_negotiation.cc:407] Connection header received
0806 12:33:29.558208 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.558212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.558266 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.558318 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.558946 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47792: BlockingRecv error: recv got EOF from 127.0.0.1:47792 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 12:33:29.595228 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.593849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.6.152.189:41481)
0806 12:33:29.594000 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.594006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.594224 (+ 218us) server_negotiation.cc:407] Connection header received
0806 12:33:29.594415 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.594419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.594471 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.594566 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.595071 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47808: BlockingRecv error: recv got EOF from 127.0.0.1:47808 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 12:33:29.662534 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.661457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.6.152.189:41481)
0806 12:33:29.661603 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.661607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.661725 (+ 118us) server_negotiation.cc:407] Connection header received
0806 12:33:29.661849 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.661852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.661901 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.662001 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.662432 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47816: BlockingRecv error: recv got EOF from 127.0.0.1:47816 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 12:33:29.793886 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:29.792730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.6.152.189:41481)
0806 12:33:29.792892 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:29.792900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:29.793027 (+ 127us) server_negotiation.cc:407] Connection header received
0806 12:33:29.793157 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:29.793160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:29.793212 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:29.793288 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:29.793757 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47830: BlockingRecv error: recv got EOF from 127.0.0.1:47830 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 12:33:30.053314 30115 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:30.052076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.6.152.189:41481)
0806 12:33:30.052228 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:30.052233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:30.052358 (+ 125us) server_negotiation.cc:407] Connection header received
0806 12:33:30.052546 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:30.052550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:30.052611 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:30.052700 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:30.053195 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47844: BlockingRecv error: recv got EOF from 127.0.0.1:47844 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 12:33:30.327939 30203 catalog_manager.cc:1259] Loaded cluster ID: d5ce688941db424eadb705fe81d0b8ff
I20260806 12:33:30.328028 30203 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: loading cluster ID for follower catalog manager: success
I20260806 12:33:30.329664 30203 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: acquiring CA information for follower catalog manager: success
I20260806 12:33:30.330753 30203 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a2f72def9ce47bfbf2b0b4ca63150f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:33:30.344949 30217 catalog_manager.cc:1259] Loaded cluster ID: d5ce688941db424eadb705fe81d0b8ff
I20260806 12:33:30.345042 30217 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: loading cluster ID for follower catalog manager: success
I20260806 12:33:30.346870 30217 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: acquiring CA information for follower catalog manager: success
I20260806 12:33:30.348150 30217 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72441edc2da14e85b75035660ec80953: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:33:30.571206 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:30.567491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.6.152.189:41481)
0806 12:33:30.567730 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:30.567734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:30.567808 (+ 74us) server_negotiation.cc:407] Connection header received
0806 12:33:30.567937 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:30.567952 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:30.568083 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:30.568189 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:30.568434 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.568961 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.569787 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.570008 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.570081 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:30.570363 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:30.570792 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:30.570883 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:30.570973 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 12:33:30.571019 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260806 12:33:30.577637 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:30.574262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.6.152.188:40785)
0806 12:33:30.574520 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:30.574524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:30.574538 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:33:30.574616 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:30.574618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:30.574723 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:30.574815 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:30.575010 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.575599 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.576372 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.576592 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.576664 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:30.576918 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:30.577296 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:30.577383 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:30.577473 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 12:33:30.577520 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260806 12:33:30.584833 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:30.580856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.6.152.188:40785)
0806 12:33:30.580952 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:30.580957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:30.581234 (+ 277us) server_negotiation.cc:407] Connection header received
0806 12:33:30.581426 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:30.581430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:30.581490 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:30.581584 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:30.581863 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.582443 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.583662 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.583904 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.584036 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:30.584299 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:30.584373 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:30.584460 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:30.584621 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 12:33:30.584685 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 12:33:30.585136 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:30.580802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.6.152.189:41481)
0806 12:33:30.580997 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:30.581002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:30.581375 (+ 373us) server_negotiation.cc:407] Connection header received
0806 12:33:30.581572 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:30.581577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:30.581626 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:30.581697 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:30.581973 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.582594 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.583590 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.583929 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.584026 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:30.584499 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:30.584586 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:30.584655 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:30.584952 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 12:33:30.585017 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 12:33:30.585799 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:30.580530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.6.152.190:34291)
0806 12:33:30.580856 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:30.580861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:30.580887 (+ 26us) server_negotiation.cc:407] Connection header received
0806 12:33:30.580989 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:30.580991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:30.581092 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:30.581218 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:30.581457 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.582265 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.583638 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:30.584013 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:30.584102 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:30.584544 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:30.585122 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:30.585233 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:30.585568 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 12:33:30.585659 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":70,"threads_started":1}
I20260806 12:33:30.588999 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 29965
I20260806 12:33:30.597405 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 30040
I20260806 12:33:30.604954 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 30116
2026-08-06T12:33:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3120 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 12:33:30 dist-test-slave-1xrh krb5kdc[30334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 12:33:30 dist-test-slave-1xrh krb5kdc[30334](info): set up 2 sockets
Aug 06 12:33:30 dist-test-slave-1xrh krb5kdc[30334](info): commencing operation
krb5kdc: starting...
W20260806 12:33:31.678573 6754 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/test-user.keytab.
Aug 06 12:33:31 dist-test-slave-1xrh krb5kdc[30334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T12:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T12:33:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.152.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
Entry for principal kudu/127.6.152.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab.
I20260806 12:33:31.800594 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33277
--webserver_interface=127.6.152.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:33277,127.6.152.189:46875,127.6.152.188:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 12:33:31.908707 30347 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 12:33:31.908941 30347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:31.908962 30347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:31.909020 30347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:31.912817 30347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:31.912859 30347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:31.912878 30347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:31.912894 30347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:31.912907 30347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:31.917909 30347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:33277,127.6.152.189:46875,127.6.152.188:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.190.keytab
--principal=kudu/127.6.152.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.190:33277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.152.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:31.918780 30347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:31.919739 30347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:31.927874 30353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:31.927924 30354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:31.927866 30356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:31.928366 30347 server_base.cc:1034] running on GCE node
I20260806 12:33:31.928856 30347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:31.929889 30347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:31.931051 30347 hybrid_clock.cc:648] HybridClock initialized: now 1786019611931025 us; error 40 us; skew 500 ppm
Aug 06 12:33:31 dist-test-slave-1xrh krb5kdc[30334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:31.933822 30347 init.cc:377] Logged in from keytab as kudu/127.6.152.190@KRBTEST.COM (short username kudu)
I20260806 12:33:31.934664 30347 webserver.cc:460] Webserver started at http://127.6.152.190:45697/ using document root <none> and password file <none>
I20260806 12:33:31.935258 30347 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:31.935326 30347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:31.935489 30347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:31.937391 30347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data/instance:
uuid: "eb56084d949c4cf083ba6f6b780ded80"
format_stamp: "Formatted at 2026-08-06 12:33:31 on dist-test-slave-1xrh"
server_key: "0bb0be0ba444731e652acac61016133d"
server_key_iv: "ac0295f93b166904d948c139d5abc56d"
server_key_version: "encryptionkey@0"
I20260806 12:33:31.937891 30347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal/instance:
uuid: "eb56084d949c4cf083ba6f6b780ded80"
format_stamp: "Formatted at 2026-08-06 12:33:31 on dist-test-slave-1xrh"
server_key: "0bb0be0ba444731e652acac61016133d"
server_key_iv: "ac0295f93b166904d948c139d5abc56d"
server_key_version: "encryptionkey@0"
I20260806 12:33:31.941423 30347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 12:33:31.943679 30363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:31.944839 30347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:31.944985 30347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal
uuid: "eb56084d949c4cf083ba6f6b780ded80"
format_stamp: "Formatted at 2026-08-06 12:33:31 on dist-test-slave-1xrh"
server_key: "0bb0be0ba444731e652acac61016133d"
server_key_iv: "ac0295f93b166904d948c139d5abc56d"
server_key_version: "encryptionkey@0"
I20260806 12:33:31.945092 30347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:31.969535 30347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:31.972756 30347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:31.972970 30347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:31.979889 30347 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.190:33277
I20260806 12:33:31.980211 30417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.190:33277 every 8 connection(s)
I20260806 12:33:31.980840 30347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/data/info.pb
I20260806 12:33:31.983513 30418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:31.987157 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 30347
I20260806 12:33:31.987309 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-0/wal/instance
I20260806 12:33:31.987602 6754 external_mini_cluster.cc:1428] Setting key 219a94218e6e59344f00e0ec3a3c3917
I20260806 12:33:31.987265 30418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.152.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.189@KRBTEST.COM" created.
Aug 06 12:33:31 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:32.005465 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:31.989877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33277 (local address 127.0.0.1:47590)
0806 12:33:31.990523 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:31.990571 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:31.991862 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:31.992572 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:31.992577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:31.993017 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:31.993238 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:31.993250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:31.994541 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:31.994557 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:31.995315 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:31.995324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:31.995939 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:31.995943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:31.996057 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:31.996701 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:31.996716 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.000186 (+ 3470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.002530 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.002540 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.002547 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.002772 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.003198 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.003201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.003202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.003310 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.004132 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.004138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.004261 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.004392 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.004637 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"spinlock_wait_cycles":43264,"thread_start_us":147,"threads_started":1}
I20260806 12:33:32.005470 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:31.989877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47590 (local address 127.6.152.190:33277)
0806 12:33:31.990377 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:31.990405 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:31.990684 (+ 279us) server_negotiation.cc:407] Connection header received
0806 12:33:31.992211 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:31.992216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:31.992395 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:31.992576 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:31.993387 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:31.994384 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:31.995425 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:31.995788 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:31.995966 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.000340 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.000345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.000355 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.000377 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.002411 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.002902 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.002911 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.002916 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.003069 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.003427 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.003433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.003438 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.004002 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.004155 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.004542 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.004720 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":147,"threads_started":1}
I20260806 12:33:32.008016 30418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:32.008720 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.008488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:36948)
0806 12:33:32.008602 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.008669 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 12:33:32.008968 30366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
I20260806 12:33:32.009785 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.009600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:36964)
0806 12:33:32.009695 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.009737 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 12:33:32.009979 30418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:46875: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab.
Entry for principal kudu/127.6.152.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab.
I20260806 12:33:32.017663 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:46875
--webserver_interface=127.6.152.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab
--principal=kudu/127.6.152.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:33277,127.6.152.189:46875,127.6.152.188:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:33:32.059258 30418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } attempt: 1
I20260806 12:33:32.060143 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.059829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:36974)
0806 12:33:32.059973 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.060066 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 12:33:32.060855 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.060648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:36976)
0806 12:33:32.060749 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.060800 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 12:33:32.060979 30418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:46875: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
W20260806 12:33:32.127364 30425 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 12:33:32.127655 30425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:32.127722 30425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:32.127800 30425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 12:33:32.129132 30418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } attempt: 2
I20260806 12:33:32.129949 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.129612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:36990)
0806 12:33:32.129782 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.129852 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 12:33:32.130672 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.130443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:36998)
0806 12:33:32.130556 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.130607 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 12:33:32.130800 30418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.189:46875: Network error: Client connection negotiation failed: client connection to 127.6.152.189:46875: connect: Connection refused (error 111)
W20260806 12:33:32.132304 30425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:32.132349 30425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:32.132366 30425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:32.132383 30425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:32.132397 30425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:32.137447 30425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:33277,127.6.152.189:46875,127.6.152.188:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.189.keytab
--principal=kudu/127.6.152.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.189:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.152.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:32.138329 30425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:32.139271 30425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 12:33:32.147109 30430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:32.147265 30431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:32.147099 30433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:32.147493 30425 server_base.cc:1034] running on GCE node
I20260806 12:33:32.147977 30425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:32.149034 30425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:32.150203 30425 hybrid_clock.cc:648] HybridClock initialized: now 1786019612150185 us; error 28 us; skew 500 ppm
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:32.152829 30425 init.cc:377] Logged in from keytab as kudu/127.6.152.189@KRBTEST.COM (short username kudu)
I20260806 12:33:32.153663 30425 webserver.cc:460] Webserver started at http://127.6.152.189:34833/ using document root <none> and password file <none>
I20260806 12:33:32.154182 30425 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:32.154269 30425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:32.154503 30425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:32.156428 30425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data/instance:
uuid: "933d17fdee4947dcba67a8cb681ebdb9"
format_stamp: "Formatted at 2026-08-06 12:33:32 on dist-test-slave-1xrh"
server_key: "ac2244d360bd3edf1d68e0e1a2bcc892"
server_key_iv: "1134b8ac0424766b1be97854edb432d0"
server_key_version: "encryptionkey@0"
I20260806 12:33:32.156981 30425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal/instance:
uuid: "933d17fdee4947dcba67a8cb681ebdb9"
format_stamp: "Formatted at 2026-08-06 12:33:32 on dist-test-slave-1xrh"
server_key: "ac2244d360bd3edf1d68e0e1a2bcc892"
server_key_iv: "1134b8ac0424766b1be97854edb432d0"
server_key_version: "encryptionkey@0"
I20260806 12:33:32.160365 30425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:33:32.162561 30440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:32.163733 30425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 12:33:32.163882 30425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal
uuid: "933d17fdee4947dcba67a8cb681ebdb9"
format_stamp: "Formatted at 2026-08-06 12:33:32 on dist-test-slave-1xrh"
server_key: "ac2244d360bd3edf1d68e0e1a2bcc892"
server_key_iv: "1134b8ac0424766b1be97854edb432d0"
server_key_version: "encryptionkey@0"
I20260806 12:33:32.163986 30425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:32.178241 30425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:32.180858 30425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:32.181064 30425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:32.188037 30425 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.189:46875
I20260806 12:33:32.188331 30494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.189:46875 every 8 connection(s)
I20260806 12:33:32.188939 30425 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/data/info.pb
I20260806 12:33:32.191545 30495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:32.193493 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 30425
I20260806 12:33:32.193642 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-1/wal/instance
I20260806 12:33:32.193992 6754 external_mini_cluster.cc:1428] Setting key 86086ef94a9714f53742cacb8896e2b8
I20260806 12:33:32.195458 30495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.152.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.152.188@KRBTEST.COM" created.
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:32.211056 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.197258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.6.152.190:33277)
0806 12:33:32.197409 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.197413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.198313 (+ 900us) server_negotiation.cc:407] Connection header received
0806 12:33:32.199319 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.199323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.199391 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.199491 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.200626 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.201168 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.202290 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.202566 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.202687 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.206522 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.206528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.206535 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.206571 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.208673 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.209368 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.209374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.209376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.209486 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.209896 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.209898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.209899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.210068 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.210146 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.210813 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.210917 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 12:33:32.212224 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.197674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33277 (local address 127.0.0.1:47596)
0806 12:33:32.198156 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.198197 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.199066 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.199669 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.199676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.200039 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.200467 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.200488 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.201307 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.201310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.202159 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.202169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.202725 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.202729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.202896 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.203489 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.203506 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.206329 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.208795 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.208803 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.208819 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.209230 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.209589 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.209595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.209599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.209799 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.210182 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.210191 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.210539 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.210768 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.211180 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":138,"threads_started":1}
I20260806 12:33:32.213749 30495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:33:32.221954 30418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.152.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab.
Entry for principal kudu/127.6.152.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab.
I20260806 12:33:32.222823 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.214227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:37000)
0806 12:33:32.214351 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.214363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.214425 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.215069 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.215072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.215245 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.215372 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.215378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.216378 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.216381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.216957 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.216966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.217602 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.217605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.217677 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.218153 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.218164 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.219337 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.221126 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.221128 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.221130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.221379 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.221672 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.221675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.221676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.221751 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.222299 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.222306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.222416 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.222540 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.222672 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 12:33:32.223393 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.214337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.6.152.189:46875)
0806 12:33:32.214654 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.214663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.214682 (+ 19us) server_negotiation.cc:407] Connection header received
0806 12:33:32.214716 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.214720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.214936 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.215069 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.215508 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.216274 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.217061 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.217501 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.217586 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.219495 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.219503 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.219512 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.219551 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.221028 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.221484 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.221489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.221490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.221563 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.221831 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.221834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.221835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.222182 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.222322 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.223067 (+ 745us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.223260 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":149,"threads_started":1}
I20260806 12:33:32.223373 6754 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknpcosN/build/debug/bin/kudu
/tmp/dist-test-tasknpcosN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:36577
--webserver_interface=127.6.152.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab
--principal=kudu/127.6.152.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.152.148:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.152.190:33277,127.6.152.189:46875,127.6.152.188:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 12:33:32.225627 30495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:32.226331 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.226092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44928)
0806 12:33:32.226202 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.226278 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 12:33:32.227301 30442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
I20260806 12:33:32.228124 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.227914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44942)
0806 12:33:32.228023 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.228074 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
W20260806 12:33:32.228343 30495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:36577: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
I20260806 12:33:32.231988 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.222786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.6.152.189:46875)
0806 12:33:32.223330 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.223339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.223353 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:33:32.223391 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.223397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.223451 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.223594 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.224106 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.224969 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.225880 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.226294 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.226511 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.228878 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.228881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.228883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.228903 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.230289 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.230811 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.230813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.230814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.230885 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.231211 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.231214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.231216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.231380 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.231455 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.231793 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.231879 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":98,"threads_started":1}
I20260806 12:33:32.232229 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.222401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:37002)
0806 12:33:32.222568 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.222585 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.222664 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.223653 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.223659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.223852 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.223976 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.223984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.225095 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.225099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.225756 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.225767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.226425 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.226428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.226542 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.226998 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.227018 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.228728 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.230402 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.230409 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.230413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.230701 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.230980 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.230985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.230990 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.231096 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.231489 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.231494 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.231617 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.231747 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.231905 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 12:33:32.233028 30418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } has no permanent_uuid. Determining permanent_uuid...
I20260806 12:33:32.233736 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.233444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44948)
0806 12:33:32.233586 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.233650 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 12:33:32.234411 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.234180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44962)
0806 12:33:32.234322 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.234363 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 12:33:32.234616 30418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:36577: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
I20260806 12:33:32.265764 30418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } attempt: 1
I20260806 12:33:32.266531 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.266263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44968)
0806 12:33:32.266400 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.266453 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 12:33:32.267310 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.267060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44974)
0806 12:33:32.267212 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.267259 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 12:33:32.267485 30418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:36577: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
I20260806 12:33:32.277621 30495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } attempt: 1
I20260806 12:33:32.278306 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.278033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44976)
0806 12:33:32.278169 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.278231 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 12:33:32.278986 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.278788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44988)
0806 12:33:32.278889 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.278932 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 12:33:32.279141 30495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:36577: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
W20260806 12:33:32.333868 30502 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 12:33:32.334124 30502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 12:33:32.334190 30502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 12:33:32.334280 30502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 12:33:32.338301 30502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 12:33:32.338352 30502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 12:33:32.338371 30502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 12:33:32.338387 30502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 12:33:32.338402 30502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 12:33:32.342671 30418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } attempt: 2
I20260806 12:33:32.343603 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.343219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:44998)
0806 12:33:32.343414 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.343508 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 12:33:32.343909 30502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.152.148:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.152.190:33277,127.6.152.189:46875,127.6.152.188:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/krb5kdc/kudu_127.6.152.188.keytab
--principal=kudu/127.6.152.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.152.188:36577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.152.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 11:52:03 UTC on ccycloud.quasar-qsdxvn.root.comops.site
build id 4570386
I20260806 12:33:32.344533 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.344319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:45004)
0806 12:33:32.344425 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.344471 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 12:33:32.344667 30418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:36577: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
I20260806 12:33:32.345052 30502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 12:33:32.346001 30502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 12:33:32.347281 30495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } attempt: 2
I20260806 12:33:32.347944 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.347657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:45012)
0806 12:33:32.347789 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.347848 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 12:33:32.348584 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.348379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:45028)
0806 12:33:32.348487 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.348530 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 12:33:32.348706 30495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.152.188:36577: Network error: Client connection negotiation failed: client connection to 127.6.152.188:36577: connect: Connection refused (error 111)
W20260806 12:33:32.354470 30509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 12:33:32.354404 30508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:32.354562 30502 server_base.cc:1034] running on GCE node
W20260806 12:33:32.354417 30511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 12:33:32.355311 30502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 12:33:32.356345 30502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 12:33:32.357533 30502 hybrid_clock.cc:648] HybridClock initialized: now 1786019612357519 us; error 23 us; skew 500 ppm
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 12:33:32.360383 30502 init.cc:377] Logged in from keytab as kudu/127.6.152.188@KRBTEST.COM (short username kudu)
I20260806 12:33:32.361209 30502 webserver.cc:460] Webserver started at http://127.6.152.188:45087/ using document root <none> and password file <none>
I20260806 12:33:32.361696 30502 fs_manager.cc:359] Metadata directory not provided
I20260806 12:33:32.361750 30502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 12:33:32.361903 30502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 12:33:32.363901 30502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data/instance:
uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb"
format_stamp: "Formatted at 2026-08-06 12:33:32 on dist-test-slave-1xrh"
server_key: "542ab9602c64c187634596cf40e44e7d"
server_key_iv: "6d987073873eadb110e354e0f2588c60"
server_key_version: "encryptionkey@0"
I20260806 12:33:32.364475 30502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal/instance:
uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb"
format_stamp: "Formatted at 2026-08-06 12:33:32 on dist-test-slave-1xrh"
server_key: "542ab9602c64c187634596cf40e44e7d"
server_key_iv: "6d987073873eadb110e354e0f2588c60"
server_key_version: "encryptionkey@0"
I20260806 12:33:32.368150 30502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 12:33:32.370550 30519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 12:33:32.371805 30502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 12:33:32.371933 30502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data,/tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal
uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb"
format_stamp: "Formatted at 2026-08-06 12:33:32 on dist-test-slave-1xrh"
server_key: "542ab9602c64c187634596cf40e44e7d"
server_key_iv: "6d987073873eadb110e354e0f2588c60"
server_key_version: "encryptionkey@0"
I20260806 12:33:32.372061 30502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 12:33:32.382644 30502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 12:33:32.385839 30502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 12:33:32.385995 30502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 12:33:32.393057 30502 rpc_server.cc:307] RPC server started. Bound to: 127.6.152.188:36577
I20260806 12:33:32.393334 30574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.152.188:36577 every 8 connection(s)
I20260806 12:33:32.394114 30502 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/data/info.pb
I20260806 12:33:32.396962 30575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 12:33:32.400481 6754 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknpcosN/build/debug/bin/kudu as pid 30502
I20260806 12:33:32.400586 6754 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknpcosN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786019324359312-6754-0/minicluster-data/master-2/wal/instance
I20260806 12:33:32.400875 6754 external_mini_cluster.cc:1428] Setting key 7e00934a064eebad496fbce56ace6457
I20260806 12:33:32.400676 30575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:33:32.413558 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.402553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.6.152.190:33277)
0806 12:33:32.402724 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.402730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.403001 (+ 271us) server_negotiation.cc:407] Connection header received
0806 12:33:32.403711 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.403715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.403763 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.403881 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.405151 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.405638 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.406675 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.406891 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.407000 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.409660 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.409664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.409666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.409689 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.411372 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.412074 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.412077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.412078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.412150 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.412496 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.412502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.412506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.412685 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.412792 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.413271 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.413388 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 12:33:32.414366 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.402405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.190:33277 (local address 127.0.0.1:47610)
0806 12:33:32.402848 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.402894 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.403503 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.404280 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.404284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.404657 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.405002 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.405017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.405745 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.405749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.406544 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.406554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.407054 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.407058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.407283 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.407885 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.407909 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.409482 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.411489 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.411497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.411508 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.411970 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.412245 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.412250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.412252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.412391 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.412792 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.412799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.413033 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.413214 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.413490 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":26,"spinlock_wait_cycles":12032,"thread_start_us":162,"threads_started":1}
I20260806 12:33:32.414366 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.402235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.6.152.188:36577)
0806 12:33:32.402647 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.402665 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.402706 (+ 41us) server_negotiation.cc:407] Connection header received
0806 12:33:32.403488 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.403505 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.403745 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.403919 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.404649 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.405856 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.406707 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.407073 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.407296 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.410093 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.410099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.410108 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.410129 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.411985 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.412448 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.412453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.412455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.412539 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.412825 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.412829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.412831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.413085 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.413283 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.413524 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.413658 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":265,"spinlock_wait_cycles":768,"thread_start_us":116,"threads_started":1}
I20260806 12:33:32.415665 30575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.190@KRBTEST.COM
I20260806 12:33:32.424345 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.416168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.189:46875 (local address 127.0.0.1:37014)
0806 12:33:32.416323 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.416336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.416404 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.416791 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.416794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.417019 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.417146 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.417156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.417968 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.417971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.418598 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.418609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.419056 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.419062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.419211 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.419620 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.419633 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.420946 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.422853 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.422859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.422861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.423058 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.423367 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.423371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.423373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.423462 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.423857 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.423860 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.423958 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.424065 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.424186 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 12:33:32.424376 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.416276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37014 (local address 127.6.152.189:46875)
0806 12:33:32.416410 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.416415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.416432 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:33:32.416509 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.416511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.416579 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.416662 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.417273 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.417860 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.418713 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.418947 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.419022 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.421087 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.421090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.421093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.421117 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.422738 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.423179 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.423183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.423185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.423273 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.423554 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.423559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.423561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.423740 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.423865 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.424119 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.424234 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 12:33:32.425091 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.416709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.6.152.190:33277)
0806 12:33:32.416824 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.416827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.416839 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:33:32.416959 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.416961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.417007 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.417065 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.417688 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.418234 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.419156 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.419412 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.419523 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.421887 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.421891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.421893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.421917 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.423285 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.423697 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.423701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.423703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.423802 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.424090 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.424094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.424095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.424287 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.424726 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.424808 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.424929 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 12:33:32.425171 30575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.188@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.152.189@KRBTEST.COM
I20260806 12:33:32.433835 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.425605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:45052)
0806 12:33:32.425711 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.425723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.425811 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.426128 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.426132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.426293 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.426389 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.426397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.427246 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.427248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.428028 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.428039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.428485 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.428488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.428543 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.428960 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.428973 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.430457 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.432209 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.432214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.432220 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.432492 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.432782 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.432788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.432792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.432887 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.433292 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.433302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.433390 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.433508 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.433660 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 12:33:32.434154 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.425819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.6.152.188:36577)
0806 12:33:32.425916 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.425920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.425933 (+ 13us) server_negotiation.cc:407] Connection header received
0806 12:33:32.425967 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.425971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.426008 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.426080 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.426514 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.427093 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.428142 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.428370 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.428486 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.430590 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.430593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.430594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.430614 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.432069 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.432612 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.432615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.432616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.432684 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.433000 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.433004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.433005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.433193 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.433266 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.433553 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.433782 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":32}
I20260806 12:33:32.435554 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.426419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.6.152.189:46875)
0806 12:33:32.426559 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.426562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.426574 (+ 12us) server_negotiation.cc:407] Connection header received
0806 12:33:32.426604 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.426606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.426643 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.426722 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.427323 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.428045 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.429171 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.429528 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.429607 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.432050 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.432054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.432056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.432086 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.433752 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.434264 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.434269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.434273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.434364 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.434652 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.434655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.434656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.434898 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.435138 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.435274 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.435394 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 12:33:32.437985 30575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: Bootstrap starting.
I20260806 12:33:32.438956 6754 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 12:33:32.439879 30495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } attempt: 3
I20260806 12:33:32.440973 30575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:32.440339 30391 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "eb56084d949c4cf083ba6f6b780ded80"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47620
I20260806 12:33:32.441847 30575 log.cc:824] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:32.443807 30418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } attempt: 3
I20260806 12:33:32.444082 30575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: No bootstrap required, opened a new log
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019612, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.189@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:33:32.448751 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.440239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:45058)
0806 12:33:32.440353 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.440368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.440449 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.440729 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.440732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.440889 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.440965 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.440970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.441636 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.441640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.442264 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.442275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.442845 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.442848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.442960 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.443434 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.443460 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.444845 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.447221 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.447224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.447226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.447470 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.447756 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.447759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.447761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.447856 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.448249 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.448252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.448345 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.448450 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.448593 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 12:33:32.448758 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.440459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.6.152.188:36577)
0806 12:33:32.440552 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.440555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.440566 (+ 11us) server_negotiation.cc:407] Connection header received
0806 12:33:32.440600 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.440602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.440631 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.440714 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.441073 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.441529 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.442386 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.442701 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.442806 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.445430 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.445433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.445436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.445458 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.447074 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.447565 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.447569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.447571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.447656 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.447960 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.447962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.447963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.448163 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.448268 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.448519 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.448635 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 12:33:32 dist-test-slave-1xrh krb5kdc[30334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786019611, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.152.190@KRBTEST.COM for kudu/127.6.152.188@KRBTEST.COM
I20260806 12:33:32.448757 30575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.449582 30575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e60f88a3b3e4d9ea8bb0706f77855fb, State: Initialized, Role: FOLLOWER
I20260806 12:33:32.450006 30575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.450532 30582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.450670 30582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.451426 30575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:32.452287 30495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: Bootstrap starting.
I20260806 12:33:32.454921 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.444193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.152.188:36577 (local address 127.0.0.1:45064)
0806 12:33:32.444366 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 12:33:32.444380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 12:33:32.444461 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 12:33:32.444957 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 12:33:32.444960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 12:33:32.445139 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 12:33:32.445246 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.445253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.446114 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.446120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.446769 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 12:33:32.446778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.447346 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.447350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 12:33:32.447437 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.448008 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 12:33:32.448023 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 12:33:32.449485 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 12:33:32.452767 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.452771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.452773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.453031 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.453559 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 12:33:32.453564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 12:33:32.453568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 12:33:32.453676 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 12:33:32.454379 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 12:33:32.454385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 12:33:32.454503 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 12:33:32.454609 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 12:33:32.454741 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 12:33:32.455737 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.444228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.6.152.188:36577)
0806 12:33:32.444728 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.444741 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.444755 (+ 14us) server_negotiation.cc:407] Connection header received
0806 12:33:32.444790 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.444796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.444836 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.444926 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.445387 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.446006 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.446901 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.447231 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.447332 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.449751 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.449756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.449781 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.449811 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.452615 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.453182 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.453214 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.453217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.453380 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.453842 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.453846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.453848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.454194 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.454373 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.454712 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.454892 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":84,"threads_started":1}
I20260806 12:33:32.455962 30495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: Neither blocks nor log segments found. Creating new log.
W20260806 12:33:32.456538 30593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:32.456614 30593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:32.455245 30502 master_runner.cc:186] Error getting master registration for 127.6.152.190:33277: OK, instance_id { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" instance_seqno: 1786019611977965 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 12:33:32.458648 30418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80: Bootstrap starting.
I20260806 12:33:32.458671 30502 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:33:32.459980 30495 log.cc:824] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:32.461126 30418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80: Neither blocks nor log segments found. Creating new log.
I20260806 12:33:32.461910 30418 log.cc:824] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80: Log is configured to *not* fsync() on all Append() calls
I20260806 12:33:32.462038 30495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: No bootstrap required, opened a new log
I20260806 12:33:32.463888 30418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80: No bootstrap required, opened a new log
I20260806 12:33:32.466130 30495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.466329 30495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933d17fdee4947dcba67a8cb681ebdb9, State: Initialized, Role: FOLLOWER
I20260806 12:33:32.466643 30495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.466771 30418 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.466996 30418 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb56084d949c4cf083ba6f6b780ded80, State: Initialized, Role: FOLLOWER
I20260806 12:33:32.467319 30418 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.468042 30495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:32.467940 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.468302 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.468514 30418 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 12:33:32.468631 30391 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 12:33:32.468734 30391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:32.470036 30391 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.470981 30391 leader_election.cc:288] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [CANDIDATE]: Term 1 election: Requested vote from peers 933d17fdee4947dcba67a8cb681ebdb9 (127.6.152.189:46875), 0e60f88a3b3e4d9ea8bb0706f77855fb (127.6.152.188:36577)
I20260806 12:33:32.472071 30547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb56084d949c4cf083ba6f6b780ded80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb"
I20260806 12:33:32.472293 30547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 12:33:32.473634 30547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb56084d949c4cf083ba6f6b780ded80 in term 1.
I20260806 12:33:32.474066 30468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb56084d949c4cf083ba6f6b780ded80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "933d17fdee4947dcba67a8cb681ebdb9"
I20260806 12:33:32.474273 30468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [term 0 FOLLOWER]: Advancing to term 1
W20260806 12:33:32.474433 30369 tablet.cc:2369] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 12:33:32.475361 30365 leader_election.cc:302] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e60f88a3b3e4d9ea8bb0706f77855fb, eb56084d949c4cf083ba6f6b780ded80; no voters:
I20260806 12:33:32.475956 30598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 12:33:32.476195 30598 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [term 1 LEADER]: Becoming Leader. State: Replica: eb56084d949c4cf083ba6f6b780ded80, State: Running, Role: LEADER
I20260806 12:33:32.476487 30598 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } }
I20260806 12:33:32.478132 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb56084d949c4cf083ba6f6b780ded80. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.478240 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:32.478981 30468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb56084d949c4cf083ba6f6b780ded80 in term 1.
I20260806 12:33:32.482965 30347 master_runner.cc:263] No leader master found from master eb56084d949c4cf083ba6f6b780ded80
I20260806 12:33:32.483032 30347 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 12:33:32.483382 30616 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 12:33:32.483480 30616 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 12:33:32.484105 30425 master_runner.cc:263] No leader master found from master 933d17fdee4947dcba67a8cb681ebdb9
I20260806 12:33:32.484175 30425 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 12:33:32.484572 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.484712 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.485093 30629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 12:33:32.487660 30629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 12:33:32.489518 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.475210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.6.152.190:33277)
0806 12:33:32.475656 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.475660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.475689 (+ 29us) server_negotiation.cc:407] Connection header received
0806 12:33:32.475722 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.475724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.475764 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.475841 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.481721 (+ 5880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.482404 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.483518 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.483870 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.483979 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.485763 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.485767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.485769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.485797 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.487546 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.488095 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.488100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.488103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.488192 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.488532 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.488535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.488537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.488753 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.488858 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.489232 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.489363 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377}
I20260806 12:33:32.491837 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.473433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.6.152.189:46875)
0806 12:33:32.473543 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.473550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.474247 (+ 697us) server_negotiation.cc:407] Connection header received
0806 12:33:32.474463 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.474467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.474520 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.474586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.475930 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.476661 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.477731 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.477976 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.478090 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.480516 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.480520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.480522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.480548 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.483530 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.490416 (+ 6886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.490419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.490421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.490512 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.490846 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.490849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.490851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.491069 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.491206 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.491575 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.491690 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 12:33:32.498167 30547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [term 1 FOLLOWER]: Refusing update from remote peer eb56084d949c4cf083ba6f6b780ded80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:32.498844 30598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:32.498980 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:32.489959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.6.152.188:36577)
0806 12:33:32.490073 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.490077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.490344 (+ 267us) server_negotiation.cc:407] Connection header received
0806 12:33:32.490517 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.490520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.490565 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.490615 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.491417 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.492187 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.493371 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:32.493731 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:32.493862 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:32.495405 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 12:33:32.495409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 12:33:32.495412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 12:33:32.495437 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 12:33:32.497116 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.497604 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.497607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.497608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.497697 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 12:33:32.498023 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 12:33:32.498026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 12:33:32.498028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 12:33:32.498237 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 12:33:32.498353 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:32.498709 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 12:33:32.498832 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 12:33:32.502681 30582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb56084d949c4cf083ba6f6b780ded80. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.502804 30582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.504176 30467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [term 1 FOLLOWER]: Refusing update from remote peer eb56084d949c4cf083ba6f6b780ded80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 12:33:32.504786 30627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 12:33:32.506373 30627 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.506492 30627 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:32.507865 30582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.507974 30582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.508405 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb56084d949c4cf083ba6f6b780ded80. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.508515 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.509305 30633 mvcc.cc:205] Tried to move back new op lower bound from 7315536332784680960 to 7315536332706840576. Current Snapshot: MvccSnapshot[applied={T|T < 7315536332784680960}]
I20260806 12:33:32.509763 30632 mvcc.cc:205] Tried to move back new op lower bound from 7315536332784680960 to 7315536332706840576. Current Snapshot: MvccSnapshot[applied={T|T < 7315536332784680960}]
I20260806 12:33:32.510340 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.510447 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80 [sys.catalog]: This master's current role is: LEADER
I20260806 12:33:32.511288 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb56084d949c4cf083ba6f6b780ded80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb56084d949c4cf083ba6f6b780ded80" member_type: VOTER last_known_addr { host: "127.6.152.190" port: 33277 } } peers { permanent_uuid: "933d17fdee4947dcba67a8cb681ebdb9" member_type: VOTER last_known_addr { host: "127.6.152.189" port: 46875 } } peers { permanent_uuid: "0e60f88a3b3e4d9ea8bb0706f77855fb" member_type: VOTER last_known_addr { host: "127.6.152.188" port: 36577 } } }
I20260806 12:33:32.511394 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 12:33:32.513111 30629 catalog_manager.cc:1347] Generated new cluster ID: 678d4fca472f4d4bbaf80e24e73d932a
I20260806 12:33:32.513185 30629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 12:33:32.519805 30637 mvcc.cc:205] Tried to move back new op lower bound from 7315536332784680960 to 7315536332706840576. Current Snapshot: MvccSnapshot[applied={T|T < 7315536332784680960}]
I20260806 12:33:32.529268 30629 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 12:33:32.530488 30629 catalog_manager.cc:1504] Loading token signing keys...
I20260806 12:33:32.543058 30629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb56084d949c4cf083ba6f6b780ded80: Generated new TSK 0
I20260806 12:33:32.543843 30629 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 12:33:32.576721 30421 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.573540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.6.152.190:33277)
0806 12:33:32.573658 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.573662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.575571 (+ 1909us) server_negotiation.cc:407] Connection header received
0806 12:33:32.575809 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.575813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.575862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.575914 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.576576 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47646: BlockingRecv error: recv got EOF from 127.0.0.1:47646 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 12:33:32.578302 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.574276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.6.152.189:46875)
0806 12:33:32.574383 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.574388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.577189 (+ 2801us) server_negotiation.cc:407] Connection header received
0806 12:33:32.577627 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.577630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.577677 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.577727 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.578193 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37034: BlockingRecv error: recv got EOF from 127.0.0.1:37034 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 12:33:32.579268 30581 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.574969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.6.152.188:36577)
0806 12:33:32.575076 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.575081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.577377 (+ 2296us) server_negotiation.cc:407] Connection header received
0806 12:33:32.578529 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.578533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.578582 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.578633 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.579154 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45082: BlockingRecv error: recv got EOF from 127.0.0.1:45082 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 12:33:32.582937 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.581503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.6.152.189:46875)
0806 12:33:32.581598 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.581602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.582069 (+ 467us) server_negotiation.cc:407] Connection header received
0806 12:33:32.582281 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.582284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.582352 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.582401 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.582849 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37036: BlockingRecv error: recv got EOF from 127.0.0.1:37036 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 12:33:32.588649 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.586941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.6.152.189:46875)
0806 12:33:32.587040 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.587044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.587640 (+ 596us) server_negotiation.cc:407] Connection header received
0806 12:33:32.587924 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.587928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.587985 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.588039 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.588540 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37044: BlockingRecv error: recv got EOF from 127.0.0.1:37044 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 12:33:32.594296 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.592739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.6.152.189:46875)
0806 12:33:32.592851 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.592856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.593393 (+ 537us) server_negotiation.cc:407] Connection header received
0806 12:33:32.593583 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.593587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.593637 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.593689 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.594201 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37050: BlockingRecv error: recv got EOF from 127.0.0.1:37050 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 12:33:32.602051 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.600420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.6.152.189:46875)
0806 12:33:32.600563 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.600567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.601129 (+ 562us) server_negotiation.cc:407] Connection header received
0806 12:33:32.601319 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.601322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.601374 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.601428 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.601943 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37054: BlockingRecv error: recv got EOF from 127.0.0.1:37054 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 12:33:32.613710 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.612133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.6.152.189:46875)
0806 12:33:32.612243 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.612247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.612782 (+ 535us) server_negotiation.cc:407] Connection header received
0806 12:33:32.612989 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.612992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.613039 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.613090 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.613614 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37064: BlockingRecv error: recv got EOF from 127.0.0.1:37064 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 12:33:32.634073 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.632298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.6.152.189:46875)
0806 12:33:32.632436 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.632441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.633057 (+ 616us) server_negotiation.cc:407] Connection header received
0806 12:33:32.633277 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.633281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.633337 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.633392 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.633956 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37072: BlockingRecv error: recv got EOF from 127.0.0.1:37072 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 12:33:32.671236 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.669051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.6.152.189:46875)
0806 12:33:32.669207 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.669212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.669994 (+ 782us) server_negotiation.cc:407] Connection header received
0806 12:33:32.670255 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.670260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.670328 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.670395 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.671060 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37088: BlockingRecv error: recv got EOF from 127.0.0.1:37088 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 12:33:32.738806 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.737641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.6.152.189:46875)
0806 12:33:32.737831 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.737844 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.737956 (+ 112us) server_negotiation.cc:407] Connection header received
0806 12:33:32.738084 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.738087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.738136 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.738209 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.738659 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37092: BlockingRecv error: recv got EOF from 127.0.0.1:37092 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 12:33:32.870448 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:32.869144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.6.152.189:46875)
0806 12:33:32.869346 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:32.869352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:32.869453 (+ 101us) server_negotiation.cc:407] Connection header received
0806 12:33:32.869619 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:32.869621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:32.869672 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:32.869747 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:32.870309 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37094: BlockingRecv error: recv got EOF from 127.0.0.1:37094 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 12:33:33.130095 30501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 12:33:33.128841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.6.152.189:46875)
0806 12:33:33.128991 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:33.128997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:33.129130 (+ 133us) server_negotiation.cc:407] Connection header received
0806 12:33:33.129314 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:33.129317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:33.129368 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:33.129460 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 12:33:33.129958 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37100: BlockingRecv error: recv got EOF from 127.0.0.1:37100 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 12:33:33.457980 30593 catalog_manager.cc:1259] Loaded cluster ID: 678d4fca472f4d4bbaf80e24e73d932a
I20260806 12:33:33.458069 30593 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: loading cluster ID for follower catalog manager: success
I20260806 12:33:33.459745 30593 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: acquiring CA information for follower catalog manager: success
I20260806 12:33:33.460848 30593 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e60f88a3b3e4d9ea8bb0706f77855fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:33:33.484630 30616 catalog_manager.cc:1259] Loaded cluster ID: 678d4fca472f4d4bbaf80e24e73d932a
I20260806 12:33:33.484699 30616 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: loading cluster ID for follower catalog manager: success
I20260806 12:33:33.486255 30616 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: acquiring CA information for follower catalog manager: success
I20260806 12:33:33.487371 30616 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 933d17fdee4947dcba67a8cb681ebdb9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 12:33:33.648438 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:33.644320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.6.152.189:46875)
0806 12:33:33.644642 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:33.644648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:33.644665 (+ 17us) server_negotiation.cc:407] Connection header received
0806 12:33:33.644854 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:33.644857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:33.644992 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:33.645119 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:33.645402 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.645937 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.646813 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.647050 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.647186 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:33.647526 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:33.648002 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:33.648146 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:33.648185 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 12:33:33.648242 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260806 12:33:33.655110 30715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:33.651622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.6.152.188:36577)
0806 12:33:33.651894 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:33.651900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:33.651916 (+ 16us) server_negotiation.cc:407] Connection header received
0806 12:33:33.652044 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:33.652047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:33.652225 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:33.652321 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:33.652499 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.653085 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.653858 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.654081 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.654154 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:33.654402 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:33.654788 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:33.654879 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:33.654938 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 12:33:33.654992 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
I20260806 12:33:33.662207 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:33.658140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.6.152.189:46875)
0806 12:33:33.658250 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:33.658254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:33.658542 (+ 288us) server_negotiation.cc:407] Connection header received
0806 12:33:33.658687 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:33.658690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:33.658750 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:33.658828 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:33.659113 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.659774 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.660893 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.661230 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.661325 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:33.661689 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:33.661773 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:33.661823 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:33.662025 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 12:33:33.662084 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 12:33:33.662722 30715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:33.658050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.6.152.188:36577)
0806 12:33:33.658168 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:33.658174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:33.658876 (+ 702us) server_negotiation.cc:407] Connection header received
0806 12:33:33.659047 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:33.659051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:33.659100 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:33.659183 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:33.659497 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.660223 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.661247 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.661492 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.661605 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:33.661861 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:33.661938 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:33.662020 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:33.662519 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 12:33:33.662571 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 12:33:33.663532 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 12:33:33.657927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.6.152.190:33277)
0806 12:33:33.658316 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 12:33:33.658322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 12:33:33.658529 (+ 207us) server_negotiation.cc:407] Connection header received
0806 12:33:33.658691 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 12:33:33.658694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 12:33:33.658830 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 12:33:33.658942 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 12:33:33.659327 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.660157 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.661287 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 12:33:33.661625 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 12:33:33.661727 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 12:33:33.662032 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 12:33:33.663028 (+ 996us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 12:33:33.663112 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 12:33:33.663280 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 12:33:33.663349 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":78,"threads_started":1}
I20260806 12:33:33.666612 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 30347
I20260806 12:33:33.674923 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 30425
I20260806 12:33:33.682367 6754 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknpcosN/build/debug/bin/kudu with pid 30502
2026-08-06T12:33:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3077 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6197 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289304 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST